DOB-SAC-CBF

Introduction

This project focuses on the development and implementation of DOB-SAC-CBF, which combines Disturbance Observer (DOB), Soft Actor-Critic (SAC), and Control Barrier Functions (CBF) for safe and efficient autonomous driving.

The key components of the DOB-SAC-CBF algorithm are as follows:

1. Disturbance Observer (DOB): The DOB is employed to estimate and compensate for internal and external disturbances acting on the vehicle. By accurately estimating these disturbances, the algorithm can enhance the robustness and adaptability of the autonomous driving system in dynamic environments.

2. Soft Actor-Critic (SAC): SAC is a state-of-the-art reinforcement learning algorithm that is utilized to generate optimal control actions for the vehicle. SAC learns a policy that maximizes the expected cumulative reward, thus leading the agent to the destination.

3. Control Barrier Functions (CBF): CBF is a safety-critical mechanism integrated into the DOB-SAC-CBF algorithm to ensure the safety of the autonomous vehicle during the training process. By modifying the control actions generated by SAC using predictive CBF constraints, the algorithm guarantees that the vehicle operates within safe operating regions and avoids collisions and maintaining stability.

This project provides a joint simulation to evaluate the effectiveness of the DOB-SAC-CBF algorithm. Through extensive testing and analysis, we aim to showcase the algorithm's capability in handling challenging driving scenarios, adapting to dynamic disturbances, and maintaining safety constraints.

Environment Dependencies

To run this project successfully, please ensure that the following dependency libraries are installed in your Python environment:

• casadi
• cvxopt
• gymnasium
• torch
• matplotlib

You can install these libraries individually using pip command, for example:

pip install casadi

Running the Simulation

1. After installing all the necessary dependencies, navigate to the joint_sim directory.

2. Inside this directory, there are three Jupyter Notebook files corresponding to different simulation scenarios:

   • simplified_car.ipynb: Simulation with a simplified first-order vehicle kinematic model in a static obstacle environment.
   • random.ipynb: Simulation with bicycle kinematic model in a dynamic obstacle environment.
   • intersection.ipynb: Simulation with bicycle kinematic model in an intersection scenario.

3. Open the Notebook file for the desired scenario and execute each code block according to the instructions and prompts provided.

4. Please carefully read the comments in each code block as they help understand the functionality and execution of the code.

Some trajectories observed during training

Safe Navigation to (5, 5)

Intersection without other cars

Intersection with other cars

Intersection with dense cars