mosaic-leaderboard

AI2's Mosaic Team has created benchmark datasets for various Commonsense Understanding tasks. To keep track of progress, each dataset is associated with a leaderboard linked here: https://leaderboard.allenai.org/

This repository provides implementations for baselines and evaluation scripts for each dataset.

1. αNLI: Abductive Natural Language Inference
   1. Evaluator
   2. Random Baseline
2. VCR: Visual Commonsense Reasoning
   1. Evaluator
   2. Random Baseline
3. HellaSwag: Can a Machine Really Finish Your Sentence?
   1. Evaluator
   2. Random Baseline
4. Social IQA: Commonsense Reasoning about Social Interactions
   1. Evaluator
   2. Random Baseline
5. Physical IQA: Commonsense Reasoning about Physical Interactions
   1. Evaluator
   2. Random Baseline
6. Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
   1. Evaluator
   2. Random Baseline
7. WinoGrande: An Adversarial Winograd Schema Challenge at Scale
   1. Evaluator
   2. Random Baseline