How to enable GPU support for faster podcast generation?

[Hasnain1997-ai]

Hi there!

First of all, thank you for creating this amazing library! I’ve been using podcastfy to generate podcasts, and it works great. However, I noticed that the generation process can be quite slow for longer transcripts, and I was wondering if there’s a way to leverage GPU acceleration to speed things up.

Here’s the code I’m currently using:

from podcastfy.client import generate_podcast
audio_file = generate_podcast(
    transcript_file="./main_app/Transcript.txt", 
    is_local=True,
    longform=True
)

Questions:
Does podcastfy support GPU acceleration for faster generation? If yes, how can I enable it?
If GPU support is not currently available, are there any plans to add it in the future?
Are there any specific dependencies (e.g., CUDA, cuDNN) or configurations required to use the library with a GPU?

Any guidance or suggestions would be greatly appreciated. Thank you in advance for your help!

[souzatharsis]

Hi, thanks for the question. Short answer: No. By default, we use Gemini's model. Hence, our latency is their latency, mostly. You can alternatively use local LLMs. In that case, yes, it's feasible to use local GPU hardware acceleration. However, this is not currently supported though implementation is feasible. Best Regards,

…

-- Thársis <http://linkedin.com/in/tharsissouza>

On Mon, Jan 20, 2025 at 4:03 PM Hasnain1997-ai ***@***.***> wrote: Hi there! First of all, thank you for creating this amazing library! I’ve been using podcastfy to generate podcasts, and it works great. However, I noticed that the generation process can be quite slow for longer transcripts, and I was wondering if there’s a way to leverage GPU acceleration to speed things up. Here’s the code I’m currently using: from podcastfy.client import generate_podcast audio_file = generate_podcast( transcript_file="./main_app/Transcript.txt", is_local=True, longform=True ) Questions: Does podcastfy support GPU acceleration for faster generation? If yes, how can I enable it? If GPU support is not currently available, are there any plans to add it in the future? Are there any specific dependencies (e.g., CUDA, cuDNN) or configurations required to use the library with a GPU? Any guidance or suggestions would be greatly appreciated. Thank you in advance for your help! — Reply to this email directly, view it on GitHub <#230>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADTMY3KVFSYCNY4RNMVJ7HD2LVQBJAVCNFSM6AAAAABVRD37NWVHI2DSMVQWIX3LMV43ASLTON2WKOZSHAYDAMRWGY2DCMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>