Skip to content

Commit

Permalink
demos update
Browse files Browse the repository at this point in the history
  • Loading branch information
BlackSamorez committed Mar 8, 2024
1 parent 544cb53 commit e1d1217
Showing 1 changed file with 3 additions and 10 deletions.
13 changes: 3 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,10 @@ Official PyTorch implementation for [Extreme Compression of Large Language Model

Learn how to run the prequantized models using this Google Colab examples:

Running `Mixtral` on a single T4 GPU:
| Basic AQLM <br> generation | Streaming with <br> GPU/CPU | Inference with CUDA <br> graphs (3x speedup) | Fine-tuning <br> with PEFT |
|-----------|-------|---------------|----------|
| <a target="_blank" href="https://colab.research.google.com/github/Vahe1994/AQLM/blob/main/notebooks/colab_example.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="AQLM In Colab"/></a> | <a target="_blank" href="https://colab.research.google.com/github/Vahe1994/AQLM/blob/main/notebooks/streaming_example.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="AQLM In Colab"/></a> | <a target="_blank" href="https://colab.research.google.com/github/Vahe1994/AQLM/blob/main/notebooks/aqlm_cuda_graph.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> | <a target="_blank" href="https://colab.research.google.com/github/Vahe1994/AQLM/blob/main/notebooks/aqlm_2bit_training.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> |

<a target="_blank" href="https://colab.research.google.com/github/Vahe1994/AQLM/blob/main/notebooks/colab_example.ipynb">
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="AQLM In Colab"/>
</a>

Streaming with GPU/CPU:

<a target="_blank" href="https://colab.research.google.com/github/Vahe1994/AQLM/blob/main/notebooks/streaming_example.ipynb">
<img src="https://colab.research.google.com/assets/colab-badge.svg" alt="AQLM In Colab"/>
</a>

### Models

Expand Down

0 comments on commit e1d1217

Please sign in to comment.