Open source code for AlphaFold.

A simple but complete full-attention transformer with a set of promising experimental features from various papers

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released

Reformer, the efficient Transformer, in Pytorch

📝 This repository recorded my NLP journey.

Official Pytorch implementation of (Roles and Utilization of Attention Heads in Transformer-based Neural Language Models), ACL 2020

Models and examples built with TensorFlow