TurtleSim ChatGPT

This demo demonstrates how ChatGPT can be used to call into ROS services, specifically services in turtlesim.

ROS is interfaced via WebSockets through rosbridge_suite. ChatGPT:

• Calls into WebSockets
• Does not execute any code on your machine
• Is given knowledge about the API via api.json

Prompt:

Move the turtle left by 2, then rotate 180 degrees, and move back to (5, 5). Finally, spawn a turtle named turtle2 at (10, 10) and remove turtle1.

Result:

Installation

• Have a running ROS distribution, e.g. Humble
• Clone this repo git clone [email protected]:mhubii/chatgpt_turtlesim.git
• Install Python dependencies: pip install -r requirements.txt
• Register an account at OpenAI and get a key Where do I find my Secret API Key?

Run the Demo

1. Run turtlesim

source /opt/ros/humble/setup.bash # source your ROS distribution
ros2 run turtlesim turtlesim_node

2. Run rosbridge_server

source /opt/ros/humble/setup.bash # source your ROS distribution
ros2 launch rosbridge_server rosbridge_websocket_launch.xml

3. Prompt OpenAI's GPT models

python main.py \
    --prompt "\
        Move the turtle left by 2, then rotate 180 degrees, and move back to (5, 5).\
        Finally, spawn a turtle named turtle2 at (10, 10) and remove turtle1.\
    " \
    --key your_key

You should see the turtle move.

Notes

This repository is just a proof of concept and it is rather slow. It, however, demonstrates how such a system could work more broadly across ROS.

Why roslibpy and rosbridge_suite? Why not just execute Python code?

Operating ROS services through WebSockets limits GPT's access to your system. It futher relieves users from ROS dependencies.

Where to go from here?

Play around and have ChatGPT call into any desired services.

Why not action clients instead of services?

By the time of this writing, action clients are not supported via rosbridge_suite yet, refer to RobotWebTools/rosbridge_suite#697.