PicoCreator / RWKV-LM-LoRA Public

forked from Blealtan/RWKV-LM-LoRA

Notifications You must be signed in to change notification settings
Fork 1
Star 10

RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Apache-2.0 license

10 stars 891 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 501 Commits
.github		.github
.gitignore		.gitignore
CITATION.cff		CITATION.cff
LICENSE		LICENSE
README.md		README.md

Repository files navigation

PicoCreator personal RWKV-LM-LoRA repo

The following repo is my personal experimental workspace for RWKV-LM-LoRA and its trainers.

You are probably looking for the

official infctx trainer here : https://github.com/RWKV/RWKV-infctx-trainer/
LoRA repo here instead : https://github.com/Blealtan/RWKV-LM-LoRA

About

RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Apache-2.0 license

Report repository

Releases

Packages