RL Models

The agents.py module implements a number of standard reinforcement learning (RL) agents that can be run on OpenAI gym environments.

1. Monte Carlo Methods

   • First-visit Monte Carlo updates (on-policy)
   • Incremental weighted importance sampling (off-policy)
   • Cross-entropy method (Mannor, Rubinstein, & Gat, 2003)

2. Temporal-Difference Methods

   • SARSA (on-policy) (Rummery & Niranjan, 1994)
   • Q-learning (off-policy) (Watkins, 1989)

3. Model-Based Methods

   • Dyna-Q/Dyna-Q+ with prioritized sweeping (Sutton, 1990; Moore & Atkeson, 1993)

Plots