Skip to content

Commit

Permalink
Merge pull request stochasticai#147 from stochasticai/glenn/readme
Browse files Browse the repository at this point in the history
docs: update readme
  • Loading branch information
sarthaklangde authored Apr 15, 2023
2 parents 1170710 + 6309796 commit 3b30281
Showing 1 changed file with 27 additions and 33 deletions.
60 changes: 27 additions & 33 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,8 @@
<img src=".github/stochastic_logo_light.svg#gh-light-mode-only" width="250" alt="Stochastic.ai"/>
<img src=".github/stochastic_logo_dark.svg#gh-dark-mode-only" width="250" alt="Stochastic.ai"/>
</p>
<h3 align="center">Build and control your own LLMs</h3>
<h3 align="center">Build, customize and control your own personal LLMs</h3>

___

`xturing` provides fast, efficient and simple fine-tuning of LLMs, such as LLaMA, GPT-J, GPT-2,
OPT, Cerebras-GPT, Galactica, and more.
By providing an easy-to-use interface for personalizing LLMs to your own data and application,
xTuring makes it simple to build and control LLMs.
The entire process can be done inside your computer or in your private cloud,
ensuring data privacy and security.

With `xturing` you can,
- Ingest data from different sources and preprocess them to a format LLMs can understand
- Scale from single to multiple GPUs for faster fine-tuning
- Leverage memory-efficient techniques (i.e. INT4, LoRA fine-tuning) to reduce your hardware costs by up to 90% of the time
- Explore different fine-tuning methods and benchmark them to find the best performing model
- Evaluate fine-tuned models on well-defined metrics for in-depth analysis

<br>
<p align="center">
<a href="https://pypi.org/project/xturing/">
<img src="https://img.shields.io/pypi/v/xturing?style=for-the-badge" />
Expand All @@ -32,12 +15,27 @@ With `xturing` you can,
<img src="https://img.shields.io/badge/Chat-FFFFFF?logo=discord&style=for-the-badge"/>
</a>
</p>
<br>

___

`xturing` provides fast, efficient and simple fine-tuning of LLMs, such as LLaMA, GPT-J, Galactica, and more.
By providing an easy-to-use interface for fine-tuning LLMs to your own data and application, xTuring makes it
simple to build, customize and control LLMs. The entire process can be done inside your computer or in your
private cloud, ensuring data privacy and security.

With `xturing` you can,
- Ingest data from different sources and preprocess them to a format LLMs can understand
- Scale from single to multiple GPUs for faster fine-tuning
- Leverage memory-efficient methods (i.e. INT4, LoRA fine-tuning) to reduce hardware costs by up to 90%
- Explore different fine-tuning methods and benchmark them to find the best performing model
- Evaluate fine-tuned models on well-defined metrics for in-depth analysis

<br>

## 🌟 _New feature_ - INT4 fine-tuning with LLaMA LoRA
## 🌟 INT4 fine-tuning with LLaMA LoRA

We are excited to announce the latest enhancement to our `xTuring` library: INT4 fine-tuning demo. With this update, you can fine-tune LLMs like LLaMA with LoRA architecture in INT4 precision with less than `6GB` of VRAM. This breakthrough significantly reduces memory requirements and accelerates the fine-tuning process, allowing you to achieve state-of-the-art performance with less computational resources.
We are excited to announce the latest enhancement to our `xTuring` library: INT4 fine-tuning demo. With this update, you can fine-tune LLMs like LLaMA with LoRA architecture in INT4 precision with less than `6 GB` of VRAM. This breakthrough significantly reduces memory requirements and accelerates the fine-tuning process, allowing you to achieve state-of-the-art performance with less computational resources.

More information about INT4 fine-tuning and benchmarks can be found in the [INT4 README](examples/int4_finetuning/README.md).

Expand All @@ -51,6 +49,8 @@ You can check out the [LLaMA INT4 fine-tuning example](examples/int4_finetuning/
## UI playground
<img src=".github/ui-playground2.gif" width="100%" style="margin: 0 1%;"/>

<br>

## ⚙️ Installation
```bash
pip install xturing
Expand Down Expand Up @@ -82,23 +82,17 @@ You can find the data folder [here](examples/llama/alpaca_data).

<br>


## 📚 Tutorials
- [Preparing your dataset](examples/llama/preparing_your_dataset.py)
- [Cerebras-GPT fine-tuning with LoRA and INT8](examples/cerebras/cerebras_lora_int8.ipynb) &ensp; [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1eKq3oF7dnK8KuIfsTE70Gvvniwr1O9D0?usp=sharing)
- [Cerebras-GPT fine-tuning with LoRA](examples/cerebras/cerebras_lora.ipynb) &ensp; [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1VjqQhstm5pT4EjPjx4Je7b3W2X1V3vDo?usp=sharing)
- [LLaMA with LoRA and INT8](examples/llama/llama_lora_int8.py) &ensp; [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1SQUXq1AMZPSLD4mk3A3swUIc6Y2dclme?usp=sharing)
- [LLaMA with LoRA](examples/llama/llama_lora.py)
- [LLaMA easy fine-tuning](examples/llama/llama.py)
- [GPT-J efficient fine-tuning with LoRA and INT8](examples/gptj/gptj_lora_int8.py) &ensp; [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1hB_8s1V9K4IzifmlmN2AovGEJzTB1c7e?usp=sharing)
- [LLaMA fine-tuning with LoRA and INT8](examples/llama/llama_lora_int8.py) &ensp; [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1SQUXq1AMZPSLD4mk3A3swUIc6Y2dclme?usp=sharing)
- [LLaMA fine-tuning with LoRA](examples/llama/llama_lora.py)
- [LLaMA fine-tuning](examples/llama/llama.py)
- [GPT-J fine-tuning with LoRA and INT8](examples/gptj/gptj_lora_int8.py) &ensp; [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1hB_8s1V9K4IzifmlmN2AovGEJzTB1c7e?usp=sharing)
- [GPT-J fine-tuning with LoRA](examples/gptj/gptj_lora.py)
- [Galactica fine-tuning with LoRA and INT8](examples/galactica/galactica_lora_int8.py)
- [Galactica fine-tuning with LoRA](examples/galactica/galactica_lora.py)
- [OPT fine-tuning with LoRA and INT8](examples/opt/opt_lora_int8.py)
- [OPT fine-tuning with LoRA](examples/opt/opt_lora.py)
- [GPT-2 fine-tuning with LoRA](examples/gpt2/gpt2_lora.py) &ensp; [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://drive.google.com/file/d/1Sh-ocNpKn9pS7jv6oBb_Q8DitFyj1avL/view?usp=sharing)


<br>

## 📊 Performance
Expand All @@ -124,7 +118,7 @@ Fine-tuning parameters:
| CPU | 190 GB | 10.2 GB | 14.9 GB |
| Time per epoch | 21 hours | 20 mins | 20 mins |

Please submit your performance results on other GPUs.
Contribute to this by submitting your performance results on other GPUs by creating an issue with your hardware specifications, memory consumption and time per epoch.

<br>

Expand All @@ -147,9 +141,9 @@ model = BaseModel.load("x/distilgpt2_lora_finetuned_alpaca")
## 📈 Roadmap
- [x] Support for LLaMA, GPT-J, GPT-2, OPT, Cerebras-GPT, Galactica and Bloom models
- [x] Dataset generation using self-instruction
- [x] 2x more memory-efficient fine-tuning vs LoRA and unsupervised fine-tuning
- [x] Low-precision LoRA fine-tuning and unsupervised fine-tuning
- [x] INT8 low-precision fine-tuning support
- [x] Supports OpenAI, Cohere and AI21 Studio model APIs for dataset generation
- [x] OpenAI, Cohere and AI21 Studio model APIs for dataset generation
- [x] Added fine-tuned checkpoints for some models to the hub
- [x] INT4 LLaMA LoRA fine-tuning demo
- [ ] Evaluation of LLM models
Expand Down

0 comments on commit 3b30281

Please sign in to comment.