Skip to content

Commit

Permalink
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -240,6 +240,12 @@ To add your own task:

### MTurk

An important part of ParlAI is seamless integration with Mechanical Turk for data collection, training and evaluation.
Human Turkers are also viewed as agents in ParlAI and hence person-person, person-bot, or multiple people and bots in group chat can all converse within the standard framework, switching out the roles as desired with no code changes to the agents. This is because Turkers also receive and send via a (pretty printed) version of the same interface, using the fields of the observation/action dict.
We provide two examples in the first release, collecting data, and human evaluation of a bot.

<p align=center><img width="70%" src="docs/source/\_static/img/mturk.png" /></p>

The mturk library contains the following directories and files:

- **core**: this directory contains the core code for setting up AWS backend that supports the MTurk chat interface, and code for HIT creation and approval.

0 comments on commit 471d951

Please sign in to comment.