Official Implementation of Video-MA2MBA

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Dialogue model that produces empathetic responses when trained on the EmpatheticDialogues dataset.

HumanOmni

Official implementation of USR (NeurIPS 2024)

Official Implementation of SALOVA

Visualize the intermediate output of Mistral 7B

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

Personalized Lip Reading: Adapting to Your Unique Lip Movements with Vision and Language (AAAI 2025)

This repository contains the speaker labeled information of VoxCeleb2 and LRS3 audio-visual datasets. (AAAI 2025)

Efficient Training for Multilingual Visual Speech Recognition: Pre-training with Discretized Visual Speech Representation (ACM MM 2024)

💬 An extensive collection of exceptional resources dedicated to the captivating world of talking face synthesis! ⭐ If you find this repo useful, please give it a star! 🤩

EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine

[CVPR 2024] AV2AV: Direct Audio-Visual Speech to Audio-Visual Speech Translation with Unified Audio-Visual Speech Representation

[ACL 2024 Findings] Official PyTorch Implementation code for realizing the technical part of CoLLaVO: Crayon Large Language and Vision mOdel to significantly improve zero-shot vision language perfo…

Pytorch implementation of "Towards Practical and Efficient Image-to-Speech Captioning with Vision-Language Pre-training and Multi-modal Tokens"

This is Pytorch Implementation Code for adding new features in code of Segment-Anything. Here, the features support batch-input on the full-grid prompt (automatic mask generation) with post-process…

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

multi server gpu monitoring utils

Official code for Score-Based Generative Modeling through Stochastic Differential Equations (ICLR 2021, Oral)