👻 Ghostty is a fast, feature-rich, and cross-platform terminal emulator that uses platform-native UI and GPU acceleration.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Fast Hadamard transform in CUDA, with a PyTorch interface

Parallelizing non-linear sequential models over the sequence length

An unnecessarily tiny implementation of GPT-2 in NumPy.

My personal website

The simplest, fastest repository for training/finetuning medium-sized GPTs.

Simple data balancing baselines for worst-group-accuracy benchmarks.

The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”

Train to 94% on CIFAR-10 in ~6.84 seconds on a single A100, the current world speed record. Or ~95.78% in ~114 seconds (or less!)

Implementation codes and datasets used in ICLR'22 Spotlight paper AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning.

Standard Library