Ongoing research training transformer models at scale

Code and documentation to train Stanford's Alpaca models, and generate the data.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.