This is code for the paper "Improving End-to-End Task-Oriented Dialogue System with A Simple Auxiliary Task".
Our python version is 3.6.9.
The package can be installed by running the following command.
pip install -r requirements.txt
python -m spacy download en_core_web_sm
For the experiments, we use MultiWOZ2.0 and MultiWOZ2.1.
- (MultiWOZ2.0) annotated_user_da_with_span_full.json: A fully annotated version of the original MultiWOZ2.0 data released by developers of Convlab available here.
- (MultiWOZ2.1) data.json: The original MultiWOZ 2.1 data released by researchers in University of Cambrige available here.
We use the preprocessing scripts implemented by Zhang et al., 2020. Please refer to here for the details.
python preprocess.py -version $VERSION
Our implementation supports a single GPU. Please use smaller batch sizes if out-of-memory error raises.
- MTTOD without auxiliary task (for the ablation)
python main.py -version $VERSION -run_type train -model_dir $MODEL_DIR
- MTTOD with auxiliary task
python main.py -version $VERSION -run_type train -model_dir $MODEL_DIR -add_auxiliary_task
The checkpoints will be saved at the end of each epoch (the default training epoch is set to 10).
python main.py -run_type predict -ckpt $CHECKPOINT -output $MODEL_OUTPUT -batch_size $BATCH_SIZE
All checkpoints are saved in $MODEL_DIR with names such as 'ckpt-epoch10'.
The result file ($MODEL_OUTPUT) will be saved in the checkpoint directory.
To reduce inference time, it is recommended to set large $BATCH_SIZE. In our experiemnts, it is set to 16 for inference.
We use the evaluation scripts implemented by Zhang et al., 2020.
python evaluator.py -data $CHECKPOINT/$MODEL_OUTPUT
This code is based on the released code (https://github.com/thu-spmi/damd-multiwoz/) for "Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context".
For the pre-trained language model, we use huggingface's Transformer (https://huggingface.co/transformers/index.html#).
We are grateful for their excellent works.