ROMI: Reverse Offline Model-based Imagination

This repository is the implementation of the paper Offline Reinforcement Learning with Reverse Model-based Imagination.

Setup the environment

conda create -n romi python=3.6.5
pip install -r environment/requirements.txt
pip install -e ./mopo
pip install -e ./CQL/d4rl

To run the code, you can configure the environment in bash examples in bash/. Before running the code, set current directory as PYTHONPATH in examples.

To learn reverse models

cd bash
bash train_reverse_model.sh

To train diverse rollout policy

cd bash
bash train_reverse_bc.sh

To train ROMI-BCQ

cd bash
bash train_romi_bcq.sh

To train ROMI-CQL

cd bash
bash train_romi_cql.sh