Copy from OpenRLHF

BABILong is a benchmark for LLM evaluation using the needle-in-a-haystack approach.

I'm here! 😎 Personal Home Page 👋🏠

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

Custom Ring attention implementation with flash attention

Implementation of paper Data Engineering for Scaling Language Models to 128K Context

Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.

Understand and test language model architectures on synthetic tasks.

self-implemented mamba

self-implemented mamba training

This is the repository of HaluEval, a large-scale hallucination evaluation benchmark for Large Language Models.

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

unofficial implementation of the High Fidelity Neural Audio Compression

(NeurIPS 2022) On Embeddings for Numerical Features in Tabular Deep Learning

State-of-the-art deep learning based audio codec supporting both mono 24 kHz audio and stereo 48 kHz audio.

[NeurIPS'23 Oral] Visual Instruction Tuning: LLaVA (Large Language-and-Vision Assistant) built towards GPT-4V level capabilities.

[CVPR 2023] Unofficial implementation for "VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models"

A reading list for papers on causality for natural language processing (NLP)