Introduction

This repo provides OpenAI Gym compatible environments for traffic light control scenario and a bunch of baseline methods.

Environments include single intersetion (single-agent) and multi intersections (multi-agents) with different road network and traffic flow settings.

Baselines include traditional Taffic Signal Control algorithms and reinforcement learning based methods.

LibSignal is a cross-simulator environment that provides multiple traditional and Reinforcement Learning models in traffic control tasks. Currently, we support SUMO, CityFlow, and CBEine simulation environments. Conversion between SUMO and CityFlow is carefully calibrated.

Install

Source

LibSingal provides installation from source code. Please execute the following command to install and configure our environment.

mkdir DaRL
cd DaRL
git clone [email protected]:DaRL-LibSignal/LibSignal.git

Simulator environment configuration

Though CityFlow and SUMO are stable under Windows and Linux systems, we still recommend users work under the Linux system. Currently, CBEngine is stable under the Linux system.

CityFlow Environment

To install CityFlow simulator, please follow the instruction on CityFlow Doc

sudo apt update && sudo apt install -y build-essential cmake

git clone https://github.com/cityflow-project/CityFlow.git
cd CityFlow
pip install .

To test configuration:

import cityflow
env = cityflow.Engine

SUMO Environment

To install SUMO environment, please follow the instruction on SUMO Doc

sudo apt-get install cmake python g++ libxerces-c-dev libfox-1.6-dev libgdal-dev libproj-dev libgl2ps-dev swig

git clone --recursive https://github.com/eclipse/sumo

export SUMO_HOME="$PWD/sumo"
mkdir sumo/build/cmake-build && cd sumo/build/cmake-build
cmake ../..
make -j$(nproc)

To test installation:

cd ~/DaRL/sumo/bin
./sumo

To add SUMO and traci model into system PATH, execute code below:

export SUMO_HOME=~/DaRL/sumo
export PYTHONPATH="$SUMO_HOME/tools:$PYTHONPATH"

To test configuration:

import libsumo
import traci

CBEngine

CBEngine currently works stably under the Linux system; we highly recommend users choose Linux if we plan to conduct experiments under the CBEinge simulation environment. (Currently not available)

Converter

We provide a converter to transform configurations including road net and traffic flow files across CityFlow and SUMO. More details in converter.py

To convert from CityFlow to SUMO:

python converter.py --typ c2s --or_cityflownet CityFlowNetPath --sumonet ConvertedSUMONetPath --or_cityflowtraffic CityFlowTrafficPath --sumotraffic ConvertedSUMOTrafficPath 

To convert from SUMO to CityFlow:

python converter.py --typ s2c --or_sumonet SUMONetPath --cityflownet ConvertedCityFlowNetPath --or_sumotraffic SUMOTrafficPath --cityflowtraffic ConvertedCityFlowTrafficPath --sumocfg SUMOConfigs

After running the code, the converted traffic network files, traffic flow files, and some intermediate files will be generated in the specified folder.

Requirement

Our code is based on Python version 3.9 and Pytorch version 1.11.0. For example, if your CUDA version is 11.3 you can follow the instruction on PyTorch

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113

pip install -r requirements.txt

Selective agents

We also support agents implemented based on other libraries

# Colight Geometric implementation based on default environment mentioned in Requirment

pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.11.0+cu113.html

# ppo_pfrl implementation
pip install pfrl

Detailed instrcuctions can be found on page Pytorch_geometric and PFRL. After installation, user should uncomment code in PATH ./agent/__init__.py

# from .ppo_pfrl import IPPO_pfrl
# from colight import CoLightAgent

Start

Run Model Pipline

Our library has a uniform structure that empowers users to start their experiments with just one click. Users can start an experiment by setting arguments in the run.py file and start with their customized settings. The following part is the arguments provided to customize.

python run.py

Supporing parameters:

• thread_num: number of threads for cityflow simulation

• ngpu: how many gpu resources used in this experiment

• task: task type to run

• agent: agent type of agents in RL environment

• world: simulator type

• dataset: type of dataset in training process

• path: path to configuration file

• prefix: the number of predix in this running process

• seed: seed for pytorch backend
  
  

Maintaining plan

To ensure the stability of our traffic signal testbed, we will first push new code onto dev branch, after validation, then merge it into the master branch.

UPdate index	Date	Status	Merged
MPLight implementation	July-18-2022	developed	√
Libsumo integration	August-8-2022	developed	√
Delay calculation	August-8-2022	developed	√
CoLight adaptation for heterogenous network	September-1-2022	developling
Optimize FRAP and MPLight	October-4-2022	developed	√
FRAP adaptation for irregular intersections	October-18-2022	developed	√
PettingZoo envrionment to better support MARL	Jul-18-2023	developed
RLFX Agent controlling phase and duration	Jul-18-2023	developed
Ray rllib support	Jul-18-2023	developling

Citation

A whitepaper of LibSignal accepted by NeurIPS 2022 Workshop: Reinforcement Learning for Real Life is available https://arxiv.org/abs/2211.10649, and can be cited with the following bibtex entry:

@misc{2211.10649,
  Author = {Mei, Hao and Lei, Xiaoliang Lei and Da, Longchao Da and Shi, Bin and Wei, Hua},
  Title = {LibSignal: An Open Library for Traffic Signal Control},
  Year = {2022},
  booktitle={NeurIPS Workshop on Reinforcement Learning for Real Life},
}