fix examples and add readme for them (facebookresearch#39)

README.md

@@ -206,9 +206,10 @@ Currently available within this directory:
 
 This directory contains a few particular examples of basic loops.
 
+- base_train.py: _very simple example shows the outline of a training/validation loop using the default Agent parent class_
 - display_data.py: _uses agent.repeat_label to display data from a particular task provided on the command-line_
 - display_model.py: _shows the predictions of a provided model on a particular task provided on the command-line_
-- eval_model.py: _uses agent.repeat_label to compute evaluation metrics data for a particular task provided on the command-line_
+- eval_model.py: _uses the named agent to compute evaluation metrics data for a particular task provided on the command-line_
 - build_dict.py: _build a dictionary from a particular task provided on the command-line using core.dict.DictionaryAgent_
 - memnn_luatorch_cpu: _shows a few examples of training an end-to-end memory network on a few datasets_
 - drqa: _shows how to train the attentive LSTM DrQA model of [Chen et al.](https://arxiv.org/abs/1704.00051) on SQuAD._

examples/README.md

@@ -0,0 +1,55 @@
+# ParlAI examples
+
+This directory contains a few particular examples of basic loops.
+
+- base_train.py: _very simple example shows the outline of a training/validation loop using the default Agent parent class_
+- display_data.py: _uses agent.repeat_label to display data from a particular task provided on the command-line_
+- display_model.py: _shows the predictions of a provided model on a particular task provided on the command-line_
+- eval_model.py: _uses the named agent to compute evaluation metrics data for a particular task provided on the command-line_
+- build_dict.py: _build a dictionary from a particular task provided on the command-line using core.dict.DictionaryAgent_
+- memnn_luatorch_cpu: _shows a few examples of training an end-to-end memory network on a few datasets_
+- drqa: _shows how to train the attentive LSTM DrQA model of [Chen et al.](https://arxiv.org/abs/1704.00051) on SQuAD._
+
+## Running These Examples
+
+Most of them can be run simply by typing `python {example}.py -t {task_name}`. Here are some examples:
+
+Display 10 random examples from task 1 of the "1k training examples" bAbI task:
+```bash
+python display_data.py -t babi:task1k:1
+```
+
+Run a train/valid loop with the basic agent (which prints what it receives and then says hello to the teacher, rather than learning anything) on the babi task:
+```bash
+python base_train.py -t babi:task1k:1
+```
+
+Displays 100 random examples from multi-tasking on the bAbI task and the SQuAD dataset at the same time:
+```bash
+python display_data.py -t babi:task1k:1,squad -n 100
+```
+
+Evaluate an IR baseline model on the validation set of the Movies Subreddit dataset:
+```bash
+python eval_model.py -m ir_baseline -t "#moviedd-reddit" -dt valid
+```
+
+Display the predictions of that same IR baseline model:
+```bash
+python display_model.py -m ir_baseline -t "#moviedd-reddit" -dt valid
+```
+
+Build a dictionary on a bAbI "1k training examples" task 1 and save it to /tmp/dict.tsv
+```bash
+python build_dict.py -t babi:task1k:1 --dict-savepath /tmp/dict.tsv
+```
+
+Train a simple cpu-based memory network on the "10k training examples" bAbI task 1 with 8 threads (python processes) using Hogwild (requires zmq and Lua Torch):
+```bash
+python memnn_luatorch_cpu/full_task_train.py -t babi:task10k:1 -n 8
+```
+
+Trains an attentive LSTM model on the SQuAD dataset with a batch size of 32 examples (pytorch and regex):
+```bash
+python drqa/train.py -t squad -b 32
+```

examples/base_train.py

@@ -17,7 +17,7 @@
 """
 
 from parlai.core.params import ParlaiParser
-from parlai.core.agents import Agent, Teacher
+from parlai.core.agents import Agent
 from parlai.core.worlds import create_task
 import time
 

examples/memnn_luatorch_cpu/full_task_train.py

@@ -20,6 +20,7 @@
 from parlai.core.worlds import DialogPartnerWorld, HogwildWorld
 
 import copy
+import os
 import sys
 import time
 
@@ -29,13 +30,17 @@ def main():
     argparser = ParlaiParser()
     DictionaryAgent.add_cmdline_args(argparser)
     ParsedRemoteAgent.add_cmdline_args(argparser)
+    argparser.add_argument('--num_examples', default=1000, type=int)
+    argparser.add_argument('--num_its', default=100, type=int)
+    parlai_home = os.environ['PARLAI_HOME']
     if '--remote-cmd' not in sys.argv:
         sys.argv.append('--remote-cmd')
-        sys.argv.append('luajit parlai/agents/memnn_luatorch_cpu/' +
-                        'memnn_zmq_parsed.lua')
+        sys.argv.append('luajit {}/parlai/agents/'.format(parlai_home) +
+                        'memnn_luatorch_cpu/memnn_zmq_parsed.lua')
     if '--remote-args' not in sys.argv:
         sys.argv.append('--remote-args')
-        sys.argv.append('examples/memnn_luatorch_cpu/params/params_default.lua')
+        sys.argv.append('{}/examples/'.format(parlai_home) +
+                        'memnn_luatorch_cpu/params_default.lua')
 
     opt = argparser.parse_args()
 
@@ -66,9 +71,9 @@ def main():
 
     start = time.time()
     with world_train:
-        for _ in range(100):
+        for _ in range(opt['num_its']):
             print('[ training ]')
-            for _ in range(1000 * opt.get('numthreads', 1)):
+            for _ in range(opt['num_examples'] * opt.get('numthreads', 1)):
                 world_train.parley()
             world_train.synchronize()