[ICLR 2024] Lemur: Open Foundation Models for Language Agents

🐙 OctoPack: Instruction Tuning Code Large Language Models

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

A multi-programming language benchmark for LLMs

Code and documentation to train Stanford's Alpaca models, and generate the data.

Code for the paper "Evaluating Large Language Models Trained on Code"

CLI platform to experiment with codegen. Precursor to: https://lovable.dev

A playbook for systematically maximizing the performance of deep learning models.

An implementation of a deep learning recommendation model (DLRM)

Repository for NeurIPS 2020 Spotlight "AdaBelief Optimizer: Adapting stepsizes by the belief in observed gradients"

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

PyTorch implementation of MoCo: https://arxiv.org/abs/1911.05722

Transformer implementation in PyTorch.

Automatically Discovering Fast Parallelization Strategies for Distributed Deep Neural Network Training

[ICML 2020] Obtaining Adjustable Regularization for Free via Iterate Averaging

Fair Resource Allocation in Federated Learning (ICLR '20)

Decentralized SGD and Consensus with Communication Compression: https://arxiv.org/abs/1907.09356

Synchronized Batch Normalization implementation in PyTorch.

Simple Hierarchical Count Sketch in Python

PyTorch for benchmarking communication-efficient distributed SGD optimization algorithms

A PyTorch implementation of the paper "Training Neural Networks Using Features Replay"

Communication-efficient decentralized SGD (Pytorch)

PyTorch Implementation of Momentum-Based Policy Gradient Methods

[ICLR 2020; IPDPS 2019] Fast and accurate minibatch training for deep GNNs and large graphs (GraphSAINT: Graph Sampling Based Inductive Learning Method).