RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, …

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.