In this project, an Auto Encoder (AE) and a Variational Auto Encoder (VAE) are designed in Python.
- Explore the gym framework for training RL agents.
- Apply my knowledge on VAE to learn image generation.
- Train generative models to produce sample pixel observation images from gym environments.
OpenAI's Gym is a framework for training reinforcement
learning agents. It provides a set of environments and a
standardized interface for interacting with those.
In this project, I used the CartPole environment from gym.
-
Create the env
conda create a1 python=3.8 -
Activate the env
conda activate a1 -
install torch (steps from pytorch installation guide):
-
if you don't have an nvidia gpu or don't want to bother with cuda installation:
conda install pytorch torchvision torchaudio cpuonly -c pytorch -
if you have an nvidia gpu and want to use it:
install cuda
install torch with cuda:
conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch
- other dependencies
conda install -c conda-forge matplotlib gym opencv pyglet
python3 -m pip install -r requirements.txt
MyVAE.py - my VAE model
train_vae.py - script to collect pixel observations from gym environments using a random policy, and train the VAE model
sample_vae.py - samples from the VAE trained by train_vae.py
On terminal, write:
python3 MyVae.py
python3 train_vae.py
python3 sample_vae.py
Below figure shows two images generated by sampling from the Auto Encoder (AE). It can be seen that sampling from AEs may result in garbage images. This is because AEs have a non-regularized latent space. Their entire latent space does not have a generative capability. Hence, AEs are mainly used for compression.
Variational Autoencoders (VAE), on the other hand, provide the generative capability to the entire latent space. Below figure shows two images generated by sampling from the VAE.
