Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL

Code and Data for Tau-Bench

Official Repo for Open-Reasoner-Zero

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

Fully open data curation for reasoning models

Fully open reproduction of DeepSeek-R1

This is a replicate of DeepSeek-R1-Zero and DeepSeek-R1 training on small models with limited data

A collection of LLM papers, blogs, and projects, with a focus on OpenAI o1 🍓 and reasoning techniques.

✨✨The Curse of Multi-Modalities (CMM): Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio

Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"

Long Context Extension and Generalization in LLMs

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

Qwen2.5 is the large language model series developed by Qwen team, Alibaba Cloud.

Suri: Multi-constraint instruction following for long-form text generation (EMNLP’24)

[NeurIPS'24 Spotlight, ICLR'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an …

Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".

The official implementation of "Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks"

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

Official repo for "Make Your LLM Fully Utilize the Context"

The official Meta Llama 3 GitHub site

[ICML'24 Spotlight] LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

Reading list of Instruction-tuning. A trend starts from Natrural-Instruction (ACL 2022), FLAN (ICLR 2022) and T0 (ICLR 2022).

Code for "Mixed Cross Entropy Loss for Neural Machine Translation"