Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refactor web-llm to support unified model artifact config (mlc-ai#91)
What does this PR do? 1. make mlc-llm a submodule of web-llm, and only keep the logic of web in web-llm, so we don't need to have multiple copies of code. 2. web-llm now takes in params , tokenizer, and runtime configurations from a folder in the following structure, so different models could be handled in the same format. ``` /mlc-llm-config.json /tokenizer.json /ndarray-cache.json /param_0.bin ... Schema of mlc-llm-config.json { # root to model artifacts, default to . (same as this file) # all artifacts are in the same folder as model_url "model_url": ".", # model lib name, used to look up lib in system or dylib "model_lib": "vicuna-v1-7b-q4f16_0", # model identifier, used to specify local directory "local_id": "myllama-v1-7b-q4f16_0", # list of tokenizer files, relative to model_url "tokenizer_files": ["tokenizer.model"], # ndarray files are implicitly in the same as model_url # the lists can be found in ndarray-cache.json # other parameters "conv_template": "vicuna-v1.1", "temperature": 0.9 , .. } ```
- Loading branch information