OSLO Python library stands for Open-Source Framework for Large-scale Model Optimization. This framework that provides various GPU-based optimization technologies for large-scale modeling. 3D Parallelism and Kernel Fusion which could be useful when training a large model like EleutherAI/gpt-j-6B are the key features. OSLO makes these technologies easy-to-use by magical compatibility with Hugging Face Transformers.
Requirements for building OSLO without any issues:
- CUDA 11 & nvcc
- PyTorch >= 1.11.0
- Turing or Ampere GPU
python3 setup.py install