🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

分享 GitHub 上有趣、入门级的开源项目。Share interesting, entry-level open source projects on GitHub.

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), ga…

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

State-of-the-Art Text Embeddings

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows".

End-to-End Object Detection with Transformers

Ongoing research training transformer models at scale

A PyTorch implementation of the Transformer model in "Attention is All You Need".

BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)

PyTorch code for Vision Transformers training with the Self-Supervised learning method DINO

Google AI 2018 BERT pytorch implementation

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

keras implement of transformers for humans

A concise but complete full-attention transformer with a set of promising experimental features from various papers

A TensorFlow Implementation of the Transformer: Attention Is All You Need

Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI

Deformable DETR: Deformable Transformers for End-to-End Object Detection.

Graphormer is a general-purpose deep learning backbone for molecular modeling.

Code and model for the paper "Improving Language Understanding by Generative Pre-Training"

Reformer, the efficient Transformer, in Pytorch

Longformer: The Long-Document Transformer

Conditional Transformer Language Model for Controllable Generation

Pytorch library for fast transformer implementations

Examples of using sparse attention, as in "Generating Long Sequences with Sparse Transformers"

🐥A PyTorch implementation of OpenAI's finetuned transformer language model with a script to import the weights pre-trained by OpenAI

This is an official implementation for "Video Swin Transformers".