Skip to content

Commit

Permalink
Interactive + Self-chat + Pretty display docs (facebookresearch#2289)
Browse files Browse the repository at this point in the history
* interactive  + selfchat + pretty display docs

* remove

* remove

* make the self chat task clearer
  • Loading branch information
jaseweston authored Dec 19, 2019
1 parent db30a44 commit a6b16ef
Show file tree
Hide file tree
Showing 4 changed files with 55 additions and 7 deletions.
28 changes: 21 additions & 7 deletions docs/source/tutorial_basic.rst
Original file line number Diff line number Diff line change
Expand Up @@ -358,22 +358,22 @@ Here are some examples:

.. code-block:: python
#Train a seq2seq model on the "10k training examples" bAbI task 1 with batch size of 32 examples until accuracy reaches 95% on validation (requires pytorch):
# Train a seq2seq model on the "10k training examples" bAbI task 1 with batch size of 32 examples until accuracy reaches 95% on validation (requires pytorch):
python examples/train_model.py -t babi:task10k:1 -m seq2seq -mf /tmp/model_s2s -bs 32 -vtim 30 -vcut 0.95
#Trains an attentive LSTM model on the SQuAD dataset with a batch size of 32 examples (pytorch and regex):
# Trains an attentive LSTM model on the SQuAD dataset with a batch size of 32 examples (pytorch and regex):
python examples/train_model.py -m drqa -t squad -bs 32 -mf /tmp/model_drqa
#Tests an existing attentive LSTM model (DrQA reader) on the SQuAD dataset from our model zoo:
# Tests an existing attentive LSTM model (DrQA reader) on the SQuAD dataset from our model zoo:
python examples/eval_model.py -t squad -mf "zoo:drqa/squad/model"
#Evaluate on the bAbI test set with a human agent (using the local keyboard as input):
# Evaluate on the bAbI test set with a human agent (using the local keyboard as input):
python examples/eval_model.py -m local_human -t babi:Task1k:1 -dt valid
#Evaluate an IR baseline model on the validation set of the Movies Subreddit dataset:
# Evaluate an IR baseline model on the validation set of the Movies Subreddit dataset:
python examples/eval_model.py -m ir_baseline -t "#moviedd-reddit" -dt valid
#Display the predictions of that same IR baseline model:
# Display the predictions of that same IR baseline model:
python examples/display_model.py -m ir_baseline -t "#moviedd-reddit" -dt valid
Expand All @@ -389,8 +389,22 @@ See `this tutorial <tutorial_task.html>`_ for making your own agents.
Of course every model has various parameters and hyperparameters to set in general.


Interacting with Models
^^^^^^^^^^^^^^^^^^^^^^^

One can also talk to your models! Here are examples of talking to models already in the model zoo:

.. code-block:: python
# Interact with a Poly-Encoder model on ConvAI2
python examples/interactive.py -mf zoo:pretrained_transformers/model_poly/model -t convai2
# Interact with a Wizard Of Wikipedia (Full Dialogue Retrieval Model).
python examples/interactive.py -m projects:wizard_of_wikipedia:interactive_retrieval -t wizard_of_wikipedia
**Model Zoo**

A new feature in ParlAI is that it also now maintains a *model zoo* of existing model files of agents that have been trained on tasks. See the devoted documentation section or `here for details <https://github.com/facebookresearch/ParlAI/blob/master/parlai/zoo/model_list.py>`_.
ParlAI now maintains a *model zoo* of existing model files of agents that have been trained on tasks. See the devoted documentation section or `here for details <https://github.com/facebookresearch/ParlAI/blob/master/parlai/zoo/model_list.py>`_.

The set of agents and models in the model zoo in ParlAI is continually growing from contributors.
32 changes: 32 additions & 0 deletions docs/source/tutorial_tipsntricks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,38 @@ That is, by adding a colon ":" followed by the flag name, an equals sign, and th
You can add multiple flags, all separated by ":".


Self-Chats
##########

Sometimes it is useful to generate models talking to themselves. You can do this with:

.. code-block:: python
# Self-chatting Poly-Encoder model on ConvAI2
python parlai/scripts/self_chat.py -mf zoo:pretrained_transformers/model_poly/model -t convai2:selfchat --inference topk -ne 10 --display-examples True -dt valid
The task set by '-t' (in the above case "convai2:selfchat") links to a parlAI world that handles the particular nature of interactions, see e.g. `here <https://github.com/facebookresearch/ParlAI/blob/master/parlai/tasks/convai2/worlds.py#L98>`_
or `here <https://github.com/facebookresearch/ParlAI/blob/master/parlai/tasks/wizard_of_wikipedia/worlds.py#L106>`_.
If the model does not need to run on a particular task you can also use:


.. code-block:: python
# Self-chatting Poly-Encoder model on a generic task (so e.g., no ConvAI2 personas are input)
python parlai/scripts/self_chat.py -mf zoo:pretrained_transformers/model_poly/model -t self_chat --inference topk -ne 10 --display-examples True -dt valid
Prettifying Display of Chats
############################

This handy script can prettify the display of json file of chats (sequences of parlai messages):

.. code-block:: python
# Display conversation in HTML format.
python parlai/scripts/convo_render.py projects/wizard_of_wikipedia/chat_example1.jsonl -o /tmp/chat.html
Internal Agents, Tasks and More
###############################

Expand Down
1 change: 1 addition & 0 deletions projects/wizard_of_wikipedia/chat_example1.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"assignment_id_hashed":"e5c93eaab5f7983008be1367b540f045838ee367","dialog":[{"speaker":"human_evaluator","text":"i do not know why, but I have never gotten into E-books"},{"speaker":"model","text":" Same here. I have a kindle, but I don’t use it for books. It’s nicer to hold a real book and turn pages."},{"speaker":"human_evaluator","text":"yes, i like the physical feel and smell of a real book"},{"speaker":"model","text":"I actually had a Nook e-book where you had to download the e-book after selecting it online. I grew out of it, I prefer paper books also."},{"speaker":"human_evaluator","text":"i tried e-books but I do not like reading on a screen, its disorientated for me"},{"speaker":"model","text":"Yes, reading a book on a tablet just isn’t the same!"},{"speaker":"human_evaluator","text":"plus I like owning the physical copy of a real book, I feel like I own it"},{"speaker":"model","text":"I prefer print and newsreels."}]}
1 change: 1 addition & 0 deletions projects/wizard_of_wikipedia/chat_example2.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"assignment_id_hashed":"e5c93eaab5f7983008be1367b540f045838ee367","dialog":[{"speaker":"model","text":"I love toga parties! I usually make my toga out of a bed sheet and wear my sandals."},{"speaker":"human_evaluator","text":"Me too!!! I usually use some flip flops and my blanket!"},{"speaker":"model","text":"Toga parties typically have games involving kegs and excessive drinking!"},{"speaker":"human_evaluator","text":"Yeah! And getting to wear funny outfits and a crown of leaves!"},{"speaker":"model","text":"I love me some Toga Parties!"},{"speaker":"human_evaluator","text":"I love me some too!"},{"speaker":"model","text":"Toga parties are so much fun with the Roman or Greek theme."},{"speaker":"human_evaluator","text":"Yeah! I especially like Greek themes"}]}

0 comments on commit a6b16ef

Please sign in to comment.