Ultra-Fast and Cheaper Long-Context LLM Inference

RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation

A low-latency & high-throughput serving engine for LLMs

PipeRAG: Fast Retrieval-Augmented Generation via Algorithm-System Co-design (KDD 2025)

A large-scale simulation framework for LLM inference

DRAMsim3: a Cycle-accurate, Thermal-Capable DRAM Simulator

High performance Transformer implementation in C++.

Automatically create Faiss knn indices with the most optimal similarity search parameters.

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

A proof of concept implementation of PowerTCP within Linux Kernel

ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale

Deep Contextual Video Compression

A high-throughput and memory-efficient inference and serving engine for LLMs

Library for fast text representation and classification.

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

Beta release of Masstree.

Benchmark baseline for retrieval qa applications

A place to store the CXL simulator

Graph-structured Indices for Scalable, Fast, Fresh and Filtered Approximate Nearest Neighbor Search

Port of Facebook's LLaMA model in C/C++

A library for efficient similarity search and clustering of dense vectors.

Framework for evaluating ANNS algorithms on billion scale datasets.