PacmanDQN

Deep Reinforcement Learning in Pac-man

Install 2022

I think all that's necessary to get this to run on modern/current tensorflow is to do import tensorflow.compat.v1 as tf instead of import tensorflow as tf.

Also, do conda install cudatoolkit as outlined in tensorflow/tensorflow#45930 (comment) to fix the 2022-03-13 00:55:39.372599: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory error.

To run this on Ubuntu 18.04 (WSL2 compatible), do:

1. sudo apt-get install python3.5
2. python3.5 -m pip install tensorflow==1
3. sudo apt-get install python3.5-tk
4. python3.5 pacman.py -p PacmanDQN -n 6000 -x 5000 -l smallGrid

And you should see successful training episodes! :)

Use -q to get quiet outputs, which is important.

Demo

Example usage

Run a model on smallGrid layout for 6000 episodes, of which 5000 episodes are used for training.

$ python3 pacman.py -p PacmanDQN -n 6000 -x 5000 -l smallGrid

Layouts

Different layouts can be found and created in the layouts directory

Parameters

Parameters can be found in the params dictionary in pacmanDQN_Agents.py.

Models are saved as "checkpoint" files in the /saves directory.
Load and save filenames can be set using the load_file and save_file parameters.

Episodes before training starts: train_start
Size of replay memory batch size: batch_size
Amount of experience tuples in replay memory: mem_size
Discount rate (gamma value): discount
Learning rate: lr

Exploration/Exploitation (ε-greedy):
Epsilon start value: eps
Epsilon final value: eps_final
Number of steps between start and final epsilon value (linear): eps_step

Citation

Please cite this repository if it was useful for your research:

@article{van2016deep,
  title={Deep Reinforcement Learning in Pac-man},
  subtitle={Bachelor Thesis},
  author={van der Ouderaa, Tycho},
  year={2016},
  school={University of Amsterdam},
  type={Bachelor Thesis},
  pdf={https://esc.fnwi.uva.nl/thesis/centraal/files/f323981448.pdf},
}

• van der Ouderaa, Tycho (2016). Deep Reinforcement Learning in Pac-man.

Requirements

• python==3.5.1
• tensorflow==0.8rc

Acknowledgements

DQN Framework by (made for ATARI / Arcade Learning Environment)

• deepQN_tensorflow (https://github.com/mrkulk/deepQN_tensorflow)

Pac-man implementation by UC Berkeley:

• The Pac-man Projects - UC Berkeley (http://ai.berkeley.edu/project_overview.html)