Skip to content

Commit

Permalink
Refactor web-llm to support unified model artifact config (mlc-ai#91)
Browse files Browse the repository at this point in the history
What does this PR do?

1. make mlc-llm a submodule of web-llm, and only keep the logic of web
in web-llm, so we don't need to have multiple copies of code.
2. web-llm now takes in params , tokenizer, and runtime configurations
from a folder in the following structure, so different models could be
handled in the same format.
```
/mlc-llm-config.json
/tokenizer.json
/ndarray-cache.json
/param_0.bin
...

Schema of mlc-llm-config.json

{
   # root to model artifacts, default to . (same as this file)
   # all artifacts are in the same folder as model_url
   "model_url": ".",

   # model lib name, used to look up lib in system or dylib
   "model_lib": "vicuna-v1-7b-q4f16_0",

   # model identifier, used to specify local directory
   "local_id": "myllama-v1-7b-q4f16_0",

   # list of tokenizer files, relative to model_url
   "tokenizer_files": ["tokenizer.model"],

   # ndarray files are implicitly in the same as model_url
   # the lists can be found in ndarray-cache.json
   
   # other parameters
   "conv_template": "vicuna-v1.1",
   "temperature": 0.9 ,
   ..
}
```
  • Loading branch information
jinhongyii authored May 14, 2023
1 parent bbe768e commit 9d048b3
Show file tree
Hide file tree
Showing 24 changed files with 110 additions and 4,760 deletions.
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
[submodule "3rdparty/sentencepiece-js"]
path = 3rdparty/sentencepiece-js
url = https://github.com/tqchen/sentencepiece-js
[submodule "mlc-llm"]
path = mlc-llm
url = https://github.com/mlc-ai/mlc-llm
200 changes: 0 additions & 200 deletions build.py

This file was deleted.

Loading

0 comments on commit 9d048b3

Please sign in to comment.