Skip to content

Latest commit

 

History

History
194 lines (141 loc) · 6.96 KB

README.md

File metadata and controls

194 lines (141 loc) · 6.96 KB

ESP32 WebSocket Audio Client

This firmware turns your Seed Studio XIAO ESP32-S3 (or general ESP32 WROOM Dev module) device into a WebSocket audio client, enabling real-time full-duplex audio communication with the server hosted at ../backend. It's designed to be used in interactive toys or devices to converse with your personal AI characters.

Pin Configuration

Component Seeed Studio XIAO ESP32S3 General ESP32 Dev Board
I2S Input (Microphone)
SD D9 GPIO 13
WS D7 GPIO 5
SCK GD8 GPIO 18
I2S Output (Speaker)
WS D0 GPIO 32
BCK D1 GPIO 33
DATA D2 GPIO 25
SD (shutdown) D3 N/A
Others
LED D4 GPIO 2
Button D5 GPIO 26

Firmware burning with PlatformIO

  1. Install PlatformIO IDE (Visual Studio Code extension) if you haven't already.

  2. Create a new PlatformIO project:

    • Open PlatformIO Home
    • Click "New Project"
    • Name your project (e.g., "FullDuplexWebSocketAudio")
    • Select "Espressif ESP32 Dev Module" as the board
    • Choose "Arduino" as the framework
    • Select a location for your project

  3. Replace the contents of src/main.cpp with the provided ESP32 WebSocket Audio Client code.

  4. Add the required libraries to your platformio.ini file:

    • For Seeed Studio XIAO ESP32S3

      [env:seeed_xiao_esp32s3]
platform = espressif32
board = seeed_xiao_esp32s3
framework = arduino
monitor_speed = 115200
lib_deps =
    https://github.com/tzapu/WiFiManager.git
    gilmaimon/ArduinoWebsockets@^0.5.4
    bblanchon/ArduinoJson@^7.1.0

    • For a general ESP32 Dev board

      [env:esp32dev]
platform = espressif32
board = esp32dev
framework = arduino
monitor_speed = 115200
lib_deps =
    https://github.com/tzapu/WiFiManager.git
    gilmaimon/ArduinoWebsockets @ ^0.5.3
    bblanchon/ArduinoJson @ ^7.1.0

  5. Update the WebSocket server details in the code:

    • Find the following lines in the code and update them with your information: 
      const char *websocket_server_host = "<your-server-host>"; // this is your WiFi I.P. Address
const uint16_t websocket_server_port = 8000;
const char *websocket_server_path = "/starmoon";
const char *auth_token = "<your-auth-token-here>"; // generate auth-token in your starmoon web-app in Settings

  6. Build the project:

    • Click the "PlatformIO: Build" button in the PlatformIO toolbar or run the build task.

  7. Upload the firmware:

    • Connect your ESP32 to your computer.
    • Click the "PlatformIO: Upload" button or run the upload task.

  8. Monitor the device:

    • Open the Serial Monitor to view debug output and device status.
    • You can do this by clicking the "PlatformIO: Serial Monitor" button or running the monitor task.

  9. Connect to WiFi using the WiFi Captive portal

    • It is straightforward to connect to your local Wifi network with an SSID (WiFi name) and Password.
    • Once the device is on, it acts as an Access Point to connect to a known WiFi network.
    • Find the device name "Starmoon device" in your list of local wifi networks.
    • Press "Configure Wifi" and type in your SSID and PW for your Wifi and connect.
    • The Seeed Stuido XIAO ESP32S3 should then automatically connect to your Wifi and save your Wifi details.

Usage

  1. Power on the ESP32 device.

  2. The device will automatically connect to the WiFi network as set on the Captive portal.

  3. Press the button to initiate a full-duplex WebSocket connection to the server.

  4. The LED indicates the current status:

    • Off: Not connected
    • Solid On: Connected and listening on microphone
    • Pulsing: Streaming audio output (receiving from server)

  5. Speak into the microphone to send audio to the server.

  6. The device will play audio received from the server through the speaker.

Functions

  • micTask: Handles audio input from the microphone
  • buttonTask: Manages button presses for connection control
  • ledControlTask: Controls the LED status indicator
  • handleTextMessage: Processes text messages from the server
  • handleBinaryAudio: Processes binary audio data from the server

Customization

You can modify the following parameters in the code:

  • Buffer sizes (bufferCnt, bufferLen)
  • Debounce time for the button (DEBOUNCE_TIME)

Troubleshooting

  • If you experience connection issues, check your WiFi credentials and server details.
  • Ensure all required libraries are installed and up to date.
  • Verify that the pin configuration matches your hardware setup.

Contributing

Feel free to submit issues or pull requests to improve this firmware.