Requirements

• python 3.7.4
• pytorch 1.8.1
• torchvision 0.9.1
• JAVA 1.8 (for COCO evaluation)
  
  

Installation

pip install -r requirements.txt

Evaluate on Visual Question Answering

Data are released here

To run the evaluate code of inference:

To get a fast but not that accuracy:

cd run_scripts/vqa
bash evaluate_vqa_beam.sh val # specify 'val' or 'test'

For the best evaluation result at the cost of much slower speed:

# run on each worker after the distributed configs have been correctly set following the guide in evaluate_vqa_allcand_distributed.sh
cd run_scripts/vqa
bash evaluate_vqa_allcand_distributed.sh val # specify 'val' or 'test'