多GPU场景下显存使用疑似不合理

[yangsen-cn]

我使用hf上的lmms-lab/LLaVA-Video-7B-Qwen2
使用样例代码，对我本地2M的视频进行处理
在执行cont = model.generate 时，会报如下异常
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.03 GiB. GPU 3 has a total capacty of 11.76 GiB of which 814.31 MiB is free. Including non-PyTorch memory, this process has 10.96 GiB memory in use. Of the allocated memory 8.82 GiB is allocated by PyTorch, and 1.99 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

请问有解决方法吗？感谢

我使用的机器和报错时的显存占用如下
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.107.02 Driver Version: 550.107.02 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
|=========================================+================
| 0 NVIDIA GeForce RTX 4060 Ti 4734MiB / 16380MiB
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GeForce RTX 4060 Ti 4364MiB / 16380MiB
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA GeForce RTX 3060 4348MiB / 12288MiB
+-----------------------------------------+------------------------+----------------------+
| 3 NVIDIA GeForce RTX 3060 11224MiB / 12288MiB