A high-throughput and memory-efficient inference and serving engine for LLMs

OpTree: Optimized PyTree Utilities

cuDNN fuzzers

An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).

Efficient Triton Kernels for LLM Training

Official repository for "AM-RADIO: Reduce All Domains Into One"

A native PyTorch Library for large model training

PyTorch native quantization and sparsity for training and inference

NVIDIA Math Libraries for the Python Ecosystem

A list of awesome compiler projects and papers for tensor computation and deep learning.

Accessible large language models via k-bit quantization for PyTorch.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

A central repository to keep track of the status of work on and support for free-threaded CPython (see PEP 703), with a focus on the scientific and ML/AI ecosystem

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

An innovative superfamily of fonts for code

Oceanic Next theme for neovim

JAX-Toolbox

Clean & Elegant Color Scheme inspired by Atom One and Material

WholeGraph - large scale Graph Neural Networks

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

You like pytorch? You like micrograd? You love tinygrad! ❤️

Open alternative files for the current buffer

NeoSolarized colorscheme for NeoVim with full transparency

🌙 A better color scheme for the late night coder

Interactively select and swap function arguments, list elements, and much more. Powered by tree-sitter.

A pleasant, mild, dark (n)vim theme.

Another attempt of a flat Gruvbox theme for Neovim

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

Dim inactive windows in Neovim using window-local highlight namespaces.