Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Docker base image to CUDA 12.4.1 for TensorRT compatibility #935

Open
huyyxy opened this issue Jan 26, 2025 · 0 comments · May be fixed by #936
Open

Update Docker base image to CUDA 12.4.1 for TensorRT compatibility #935

huyyxy opened this issue Jan 26, 2025 · 0 comments · May be fixed by #936

Comments

@huyyxy
Copy link

huyyxy commented Jan 26, 2025

Is your feature request related to a problem? Please describe.

Currently, the Dockerfile uses nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04 which only supports CUDA 11.8. When attempting to use TensorRT with newer GPU hardware/drivers requiring CUDA 12.x, the existing Docker environment fails to provide compatible dependencies, leading to runtime errors and compatibility issues with modern AI frameworks.

Describe the solution you'd like

Update the Docker base image to:

FROM nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04

This change will:

Provide native CUDA 12.4.1 and cuDNN support

Enable seamless TensorRT integration

Align with modern GPU driver requirements (>= 545.x)

Maintain Ubuntu 22.04 LTS compatibility

Describe alternatives you've considered

  1. Manually installing CUDA 12.4 in existing container:

Would require complex Dockerfile modifications

Risks version conflicts with base CUDA 11.8 libraries

  1. Using CUDA forward-compatibility:

May introduce unexpected behavior

Doesn't resolve underlying dependency mismatches

  1. Maintaining separate Dockerfiles:

Increases maintenance overhead

Fragments development/production environments

Additional context

  1. Validation Plan:

Verify CUDA version post-build:

nvidia-smi | grep CUDA

Confirm TensorRT availability in Python environment:

import tensorrt
print(tensorrt.__version__)  # Should return >= 10.0.1
  1. Potential Impact:

May require updating related dependency versions (e.g., PyTorch, TorchAudio)

Need to validate all existing pipeline functionality with CUDA 12.x

  1. Reference Documentation:

NVIDIA CUDA 12.4 Release Notes

TensorRT CUDA Compatibility Matrix

huyyxy added a commit to huyyxy/CosyVoice that referenced this issue Jan 26, 2025
- Upgrade base image from nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04 to nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04
- Enable CUDA 12.4 runtime environment
- Ensure TensorRT dependency compatibility
- Validation steps:
  - Verify CUDA version via nvidia-smi after build
  - Test import tensorrt in container without errors

Closes FunAudioLLM#935
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant