Skip to content

Latest commit

 

History

History
41 lines (27 loc) · 1.86 KB

download_stablelm.md

File metadata and controls

41 lines (27 loc) · 1.86 KB

Download StableLM weights

StableLM is a family of generative language models trained by StabilityAI, trained on a dataset derived from The Pile but 3x larger, for a total of 1.5 trillion tokens. Weights are released under the CC-BY-SA license.

For more info on the models, please see the StableLM repository. 3B and a 7B checkpoints have been released, both after pre-training and after instruction tuning, using a combination of Stanford's Alpaca, Nomic-AI's gpt4all, RyokoAI's ShareGPT52K datasets, Databricks labs' Dolly, and Anthropic's HH.

To see all the available checkpoints for StableLM, run:

python scripts/download.py | grep stablelm

which will print

stabilityai/stablelm-base-alpha-3b
stabilityai/stablelm-base-alpha-7b
stabilityai/stablelm-tuned-alpha-3b
stabilityai/stablelm-tuned-alpha-7b

In order to use a specific StableLM checkpoint, for instance stablelm-base-alpha-3b, download the weights and convert the checkpoint to the lit-gpt format:

pip install huggingface_hub

python scripts/download.py --repo_id stabilityai/stablelm-base-alpha-3b

python scripts/convert_hf_checkpoint.py --checkpoint_dir checkpoints/stabilityai/stablelm-base-alpha-3b

By default, the convert_hf_checkpoint step will use the data type of the HF checkpoint's parameters. In cases where RAM or disk size is constrained, it might be useful to pass --dtype bfloat16 to convert all parameters into this smaller precision before continuing.

You're done! To execute the model just run:

pip install tokenizers

python generate/base.py --prompt "Hello, my name is" --checkpoint_dir checkpoints/stabilityai/stablelm-base-alpha-3b