Hugging Face RoBERTa with Flash Attention 2

Machine Learning Engineering Open Book

YaRN: Efficient Context Window Extension of Large Language Models

A repository to get train transformers to access longer context for causal language models, most of these methods are still in testing. Try them out if you'd like but please lmk your results so we …

Implementation of Production Ready Information Retrieval System

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

Guide: Finetune GPT2-XL (1.5 Billion Parameters) and finetune GPT-NEO (2.7 B) on a single GPU with Huggingface Transformers using DeepSpeed

Solve puzzles. Learn CUDA.

Repro is a library for easily running code from published papers via Docker.

Code and Data for Evaluation WG

Must-read papers on prompt-based tuning for pre-trained language models.

Models to perform neural summarization (extractive and abstractive) using machine learning transformers and a tool to convert abstractive summarization datasets to the extractive task.

A fast and simple framework for building and running distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning libr…