Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
XianfengWu01 authored Oct 14, 2024
1 parent 30763f6 commit caa84f7
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ Visit our website: [Everlyn.ai](https://www.everlyn.ai)

Our latest open-source research is centered around three key components. First, we introduce a new approach to video compression and tokenization, designed to improve both quality and performance. Next, we present our framework for efficient autoregressive models. Finally, we share our advancements in multimodal understanding, with a focus on reducing hallucinations in large language models.

### 1. [Distribution Matching for Vector Quantization](./WassersteinVQ)
### 1. [Distribution Matching for Vector Quantization](https://github.com/Openlyn/Wasserstein-VQ)

We tackle the challenges of instability and inefficiency in vector quantization for autoregressive video models. By employing a novel distribution matching approach based on the **Wasserstein distance**, we significantly enhance codebook utilization and reduce quantization errors. This method results in more stable training and improved performance in generative video tasks.

### 2. [EfficientARV: Efficient Autoregressive Models for Image and Video Generation](./efficientARV-1)
### 2. [EfficientARV: Efficient Autoregressive Models for Image and Video Generation](https://github.com/Openlyn/ANTRP)

EfficientARV is designed to create an efficient autoregressive model for jointly generating images and videos. The project explores multiple conditional generation tasks, such as image animation, inpainting, outpainting, video prediction, and video interpolation. Additionally, it aims to integrate these generation capabilities into Multimodal Large Language Models (MLLMs) for more interactive and robust AI systems.

### 3. [ANTRP: Intervening Anchor Token - Decoding Strategy for MLLMs](./ANTRP-1)
### 3. [ANTRP: Intervening Anchor Token - Decoding Strategy for MLLMs](https://github.com/Openlyn/EfficientARV)

Lastly, we focus on improving Multimodal Large Language Models (MLLMs) by addressing the hallucination problem. Instead of penalizing summary tokens, ANTRP intervenes in the query-key parameters variance, reducing hallucinations without additional inference time. We propose the **Dynamic Token Propagation Mechanism (TAME)**, which dynamically adjusts the eigenspectrum variance of the attention weights to alleviate over-propagation of "anchor" tokens. Extensive experiments show a strong correlation between the eigenspectrum and hallucinations, with TAME significantly reducing hallucinated objects across various MLLMs.

Expand Down

0 comments on commit caa84f7

Please sign in to comment.