Multi-Camera Microscope Setup with ESP32 Cameras and USB Hubs: Communication and Power Issues

[beniroquai]

Description:

Overview:

I have built a microscope using 24 ESP32 cameras, connected to six 4-port USB hubs. These hubs are then connected to a powered 6-port USB hub. I'm attempting to read out frames using Python or Node.js, but I'm encountering several issues.

• data is "pulled from the device - e.g. a string c is sent to the camera and the ESP then sends a base64 encoded string to the receiver
• The ESPs have an LED heartbeat that blink every second
• It blinks once when a c string is received and 4x if data is sent back

Configuration Details:

• Cameras: 24x ESP32
• USB Hubs: 6x 4-port hubs connected to a 6-port main hub
• Software: Python, Node.js (with pyserial for serial communication)
• Operating Systems tested: MacOS, Windows, Linux (Jetson)

Issues Encountered:

Power:

• External USB power does not improve the situation.
• Voltage measurements:
  • After diode: 4.2V
  • Without diode: 4.7V (with power supply plugged in)
• ESP cameras show heartbeat signals.
• No significant voltage drop observed when daisy-chaining multiple USB hubs.

Signal:

• Using an oscilloscope on D+/D- of any hub causes the connection to interrupt.
• Devices are still recognized, but data communication halts.
• Possible ESP hangs.

Software:

• Tried different baud rates (115200, 500000, 2000000); no effect observed.
• Sequentially closing and opening ports does not resolve the issue.
• All ports are detectable prior to data readout attempt.

Operating System Behavior:

• MacOS: Best performance observed.
• Windows: Sometimes fails to recognize the devices.
• Linux (Jetson): Partial and incorrect data readouts.

Potential Problems Identified:

• Lack of a quartz component in the setup.

Observations:

• Cameras work when hubs are connected sequentially and power is turned on too
• When the USB connecting the host computer is plugged in, there is a voltage drop on the camera side. Subsequently, some cameras fail to recover to a working state, the loop halts, and the LED heartbeat stops.

Seeking Suggestions:

I am looking for insights or suggestions to troubleshoot and resolve these issues, particularly focusing on the signal integrity and power supply stability in such a multi-camera, multi-hub setup.
Perhaps @ethanjli has experience with this? ;)

---

Possible Error Analysis:

1. Power Supply Issues: The voltage drops and fluctuations, especially when all components are connected, suggest a power supply issue. This might be due to the cumulative power draw of all devices exceeding the capacity of your power supply or USB hubs.

2. Signal Integrity: The interruption of data communication when using an oscilloscope suggests potential signal integrity issues. This could be due to interference, inadequate shielding, or limitations in the USB hub's ability to handle multiple data streams simultaneously.

3. Software and Baud Rate: While different baud rates have been tested, there could be other software-related issues or limitations in the ESP32's ability to handle high data rates, especially when multiplied by 24 cameras.

4. Operating System Compatibility: The varying performance across different operating systems indicates potential driver or compatibility issues.

5. Hardware Limitations or Faults: The lack of a quartz component might be affecting the timing and synchronization between the devices. Additionally, there might be hardware limitations or faults in the USB hubs or cameras themselves.

[ethanjli]

To what extent do these issues appear when you have less than 24 cameras deployed? E.g. just one USB hub of cameras, or just a few cameras on each USB hub?

[beniroquai]

Thanks @ethanjli for your comment. The situation looks as follows: I'm always filling every Camera Hub (4x USB) with all slots. The picture should illustrate this:

What I have not tried yet is if I only had one camera in every 4x hub after the 7x master hub.

EDIT: Also notable, the hubs use their integrated oscillator, no external quartz have been used (yet).

[beniroquai]

Brief update: I managed to hook up external oscillation (i.e. 12MHz Quartz). Let’s see if this makes a difference. Data wires are not same length unfortunately. Unclear if this may cause problems.