Atm very simple, just view the example in scratch.sh
. Note that this .sh
script is meant for
viewing only, and not for execution. I just used .sh
to get syntax highlighting.
This example requires you to first create a Nemo-2.0 checkpoint of a tiny llama3 model using 01-cpt-toy-llama3/01-pretrain.sh. Then, you can follow the steps in scratch.sh.