This repository contains codes of the official implementation for the paper (A Partition Filter Network for Joint Entity and Relation Extraction)
In this work, we present a new framework equipped with a novel recurrent encoder named partition
filter encoder designed for multi-task learning. The encoder enforces bilateral interaction between NER and RE in two ways:
- The shared partition represents inter-task information and is equally accessible to both tasks, allowing for balanced interaction between NER and RE.
- The task partitions represent intra-task information and are formed through concerted efforts of entity and relation gates, making sure that encoding process of entity and relation features are dependent upon each other.
The experiments were performed using one single NVIDIA-RTX3090 GPU.
The dependency packages can be installed with the following command:
pip install -r requirements.txt
We evaluate our model on five datasets: [NYT, WEBNLG, ADE, ACE2005, SCIERC].
Please follow the instructions of reademe.md in each dataset folder in ./data/ for data acquisition and preprocessing.
The training command-line is listed below:
python main.py \
--data ${NYT/WEBNLG/ADE/ACE2005/SCIERC} \
--do_train \
--do_eval \
--embed_mode ${bert_cased/albert/scibert} \
--batch_size ${20 (for most datasets) /4 (for SCIERC)} \
--lr ${0.00002 (for most datasets) /0.00001 (for SCIERC)} \
--output_file ${the name of your output files, e.g. ace_test} \
--eval_metric ${micro/macro}
After training, you will obtain three files in the ./save/${output_file}/ directory:
- ${output_file}.log records the logging information.
- ${output_file}.txt records loss, NER and RE results of dev set and test set for each epoch.
- ${output_file}.pt is the saved model with best average F1 results of NER and RE in the dev set.
The evaluation command-line is listed as follows:
python eval.py \
--data ${NYT/WEBNLG/ADE/ACE2005/SCIERC} \
--eval_metric ${micro/macro} \
--model_file ${the path of saved model you want to evaluate. e.g. save/ace_test.pt} \
--embed_mode ${bert_cased/albert/scibert}
If you want to evaluate the model with customized input, please run the following code:
python inference.py \
--model_file ${the path of your saved model} \
--sent ${sentence you want to evaluate, str type restricted}
{model_file} must contain information about the datasets the model trained on (web/nyt/ade/ace/sci)
and the type of pretrained embedding the model uses (albert/bert/scibert). For example, model_file could be set as "web_bert.pt"
input:
python inference.py \
--model_file save/sci_test_scibert.pt \
--sent "In this work , we present a new framework equipped with a novel recurrent encoder
named partition filter encoder designed for multi-task learning ."
result:
entity_name: framework, entity type: Generic
entity_name: recurrent encoder, entity type: Method
entity_name: partition filter encoder, entity type: Method
entity_name: multi-task learning, entity type: Task
triple: recurrent encoder, Used-for, framework
triple: recurrent encoder, Part-of, framework
triple: recurrent encoder, Used-for, multi-task learning
triple: partition filter encoder, Hyponym-of, recurrent encoder
triple: partition filter encoder, Used-for, multi-task learning
input:
python inference.py \
--model_file save/ace_test_albert.pt \
--sent "As Williams was struggling to gain production and an audience for his work in the late 1930s ,
he worked at a string of menial jobs that included a stint as caretaker on a chicken ranch in
Laguna Beach , California . In 1939 , with the help of his agent Audrey Wood , Williams was
awarded a $1,000 grant from the Rockefeller Foundation in recognition of his play Battle of
Angels . It was produced in Boston in 1940 and was poorly received ."
result:
entity_name: Williams, entity type: PER
entity_name: audience, entity type: PER
entity_name: his, entity type: PER
entity_name: he, entity type: PER
entity_name: caretaker, entity type: PER
entity_name: ranch, entity type: FAC
entity_name: Laguna Beach, entity type: GPE
entity_name: California, entity type: GPE
entity_name: his, entity type: PER
entity_name: agent, entity type: PER
entity_name: Audrey Wood, entity type: PER
entity_name: Williams, entity type: PER
entity_name: Rockefeller Foundation, entity type: ORG
entity_name: his, entity type: PER
entity_name: Boston, entity type: GPE
triple: caretaker, PHYS, ranch
triple: ranch, PART-WHOLE, Laguna Beach
triple: Laguna Beach, PART-WHOLE, California
We provide you with pre-trained models for NYT/WEBNLG/ACE2005/SCIERC, along with recorded results of each epoch, identical with training results under the specified configurations above.
Due to limited space in google drive, 10-fold model files for ADE are not available.
After downloading the linked files below, unzip them and put ${data}_test.pt in the directory of ./save/ before running eval.py.
Also, ${data}_test.txt and ${data}_test.log recoreds the results of each epoch. You should check that out as well.
f1 results on NYT/WEBNLG/ACE2005/SCIERC:
dataset Embedding NER RE
NYT bert-base-cased 0.9575 0.9242
WEBNLG bert-base-cased 0.9803 0.9363
ACE2005 albert-xxlarge-v1 0.8903 0.6679
SCIERC scibert-uncased 0.6682 0.3844
f1 results on ADE:
10-fold 0 1 2 3 4 5 6 7 8 9 average
bert-NER 89.6 92.3 90.3 88.9 88.8 90.2 90.1 88.5 88.0 88.9 89.6
bert-RE 80.5 85.8 79.9 79.4 79.3 80.5 80.0 78.1 76.2 79.8 80.0
albert-NER 91.4 92.9 91.9 91.5 90.7 91.6 91.9 89.9 90.6 90.7 91.3
albert-RE 83.9 86.8 82.8 83.2 82.2 82.4 84.5 82.3 81.9 82.2 83.2