Value-Based Methods

This repository contains material related to Udacity's Value-based Methods course.

Table of Contents

Tutorials

The tutorials lead you through implementing various algorithms in reinforcement learning. All of the code is in PyTorch (v0.4) and Python 3.

• Deep Q-Network: Explore how to use a Deep Q-Network (DQN) to navigate a space vehicle without crashing.

Labs / Projects

The labs and projects can be found below. All of the projects use rich simulation environments from Unity ML-Agents.

• Navigation: In the first project, you will train an agent to collect yellow bananas while avoiding blue bananas.

Resources

• Cheatsheet: You are encouraged to use this PDF file to guide your study of reinforcement learning.

OpenAI Gym Benchmarks

Box2d

• LunarLander-v2 with Deep Q-Networks (DQN) | solved in 1504 episodes

Dependencies

To set up your python environment to run the code in this repository, follow the instructions below.

1. Create (and activate) a new environment with Python 3.6.

   • Linux or Mac:

   conda create --name drlnd python=3.6
   source activate drlnd

   • Windows:

   conda create --name drlnd python=3.6 
   activate drlnd

2. Follow the instructions in this repository to perform a minimal install of OpenAI gym.

   • Install the box2d environment group by following the instructions here.

3. Clone the repository (if you haven't already!), and navigate to the python/ folder. Then, install several dependencies.

git clone https://github.com/udacity/Value-based-methods.git
cd Value-based-methods/python
pip install .

4. Create an IPython kernel for the drlnd environment.

python -m ipykernel install --user --name drlnd --display-name "drlnd"

5. Before running code in a notebook, change the kernel to match the drlnd environment by using the drop-down Kernel menu.

Want to learn more?

Come learn with us in the Deep Reinforcement Learning Nanodegree program at Udacity!