Template repo for Python projects, especially those focusing on machine learning and/or deep learning.

A very simple BERT implementation in PyTorch, which only depends on PyTorch itself.

Simple but robust implementation of LoRA for PyTorch. Compatible with NLP, CV, and other model types. Strongly typed and tested.

(Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints" (https://arxiv.org/pdf/2305.13245.pdf)

RAG application to answer questions about PDF documents using LLMs.

A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (https://arxiv.org/pdf/2307.08621.pdf)

PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)

Implementation of 1D, 2D, and 3D FFT convolutions in PyTorch. Much faster than direct convolutions for large kernel sizes.

(Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens" (https://arxiv.org/abs/2307.02486)

Code implementation from my blog post: https://fkodom.substack.com/p/transformers-from-scratch-in-pytorch

Generate text captions for images from their embeddings.

A simple, modern alternative to 'pycocotools'.

Build batch pipelines in Python that run anywhere.

Pretrained BLIP with a similar API to CLIP.

Code samples to accompany my "Intro to Deep Learning" article series on Medium.

Fastest Wordle solver in the West.

Fastest 'Semantle' solver this side of the Mississippi.

PyTorch implementation of BYOL: a fantastically simple method for self-supervised image representation learning with SOTA performance.

Minimalist library for training RL agents in PyTorch. Implements many common training algorithms, with a focus on actor-critic methods. Includes SAC, TD3, PPO, A2C, VPG.

Personal repo recreating Noise Contrastive Score Networks (https://arxiv.org/abs/1907.05600v2) for learning purposes.

Implementation of the Hindsight Experience Replay (HER) algorithm using PyTorch. Utilizes Deep Deterministic Policy Gradients for off-policy optimization of RL agents in continuous action spaces.