You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for creating this library! We are using ctransformers at PostgresML to support GGUF models from Huggingface. We need to detect and recover from CUDA out of memory errors. Currently, it appears that ctransformers exits the process whenever a CUDA error is encountered.
Is there any plan to add error handling to ctransformers? Ideally, we would get some sort of Exception in Python that we can handle.
The text was updated successfully, but these errors were encountered:
Thanks for creating this library! We are using ctransformers at PostgresML to support GGUF models from Huggingface. We need to detect and recover from CUDA out of memory errors. Currently, it appears that ctransformers exits the process whenever a CUDA error is encountered.
Is there any plan to add error handling to ctransformers? Ideally, we would get some sort of
Exception
in Python that we can handle.The text was updated successfully, but these errors were encountered: