DRL-for-mobile-robots

Implementing Deep Reinforcement Learning for mobile robot navigation using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm in a ROS2 Gazebo simulation environment. The robot learns to navigate to random goal points while avoiding obstacles, utilizing laser sensor data for obstacle detection and polar coordinates for goal specification. The training process is conducted in ROS2 Gazebo using PyTorch, with testing performed in ROS2 Foxy on Ubuntu 20.04, Python 3.8.10, and PyTorch 1.10.

Training example (Notice how robot fails to reach goal and collides almost every time):

Testing example (Notice how robot reaches goal state almost every time):

Project Report

Some more information about the implementation is available here

Installation

Main dependencies:

• ROS2 Foxy
• PyTorch
• Tensorboard
• Gazebo

Clone the repository:

$ cd ~
### Clone this repo
$ git clone https://github.com/Kanishk-Kaushal/DRL-for-mobile-robots

The network can be run with a standard 2D laser, but this implementation uses a simulated 3D Velodyne sensor

Clean the previous workspace:

$ cd ~/DRL-for-mobile-robots
$ rm -rf build install log
$ colcon clean
$ source /opt/ros/foxy/setup.bash

Compile the workspace:

$ cd ~/DRL-for-mobile-robots
### Compile
$ colcon build

Open a terminal and set up sources:

$ cd
$ source /opt/ros/foxy/setup.bash
$ source DRL-for-mobile-robots/install/setup.bash

Run the training:

$ ros2 launch td3 training_simulation.launch.py

To check the training process on tensorboard:

1. Navigate to src/td3/scripts/train_velodyne.py
2. Open this file in Visual Studio
3. Click on launch Tensorboard application

To kill the training process:

$ [Ctrl+C]

Once training is completed (Took me ~7500 episodes), test the model:

$ ros2 launch td3 test_simulation.launch.py

Gazebo environment:

Rviz:

Future scope:

To-do a comprehensive comparative analysis between using the conventional approaches and TD3 for autonomous navigation of mobile robot in indoor environments.