shumingma
Follow
Stars
Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.
SuperPrompt is an attempt to engineer prompts that might help us understand AI agents.
The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
The official Python library for the OpenAI API
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Development repository for the Triton language and compiler
Fast and memory-efficient exact attention
Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackable.
A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB team.
Official implementations for (1) BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation and (2) Discourse Centric Evaluation of Machine Translation with a Densely Annotated P…
A simple and readable neural machine translation system
Hackable and optimized Transformers building blocks, supporting a composable construction.
LightSeq: A High Performance Library for Sequence Processing and Generation
PyTorch extensions for high performance and large scale training.
Efficient, check-pointed data loading for deep learning with massive data sets.
Improved Sentence Alignment in Linear Time and Space
Collection of Evaluation Metrics and Algorithms for Machine Translation