How can we apply batch size when inference?

[cihankaradogan]

I wanna speed the inference up. I thought generating frames batch by batch can speed it up. How can I implement?

[Elsaam2y]

You can try a different model for audio features extraction and also train for lower face mask if you want to achieve real-time performance. I played with the optimization a bit here: https://github.com/Elsaam2y/DINet_optimized
and this can achieve 50-60% reduction in inference latency. Still not in real-time but could be done next.