Skip to content

Commit

Permalink
Merge pull request janhq#1021 from janhq/importModelDocs
Browse files Browse the repository at this point in the history
docs: second half of "import model docs" PR
  • Loading branch information
freelerobot authored Dec 15, 2023
2 parents 45ac606 + 0ecd06e commit 79f4bd1
Showing 1 changed file with 17 additions and 11 deletions.
28 changes: 17 additions & 11 deletions docs/docs/guides/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Navigate to `~/jan/models` folder on your computer.

In `App Settings`, go to `Advanced`, then `Open App Directory`.

Or, you can directly cd into:

```sh
# Windows
C:/Users/<your_user_name>/jan/models
Expand All @@ -35,21 +37,21 @@ jan/models
In the `models` folder, create a folder with the name of the model.

```sh
mkdir pandora-v1-q4
mkdir trinity-v1-7b
```

## 2. Create a model JSON

Jan follows a standardized model template, called a `model.json`. This allows for easy model configurations, exporting, and sharing.
Jan follows a folder-based, [standard model template](/specs/models) called a `model.json`. This allows for easy model configurations, exporting, and sharing.

```sh
cd pandora-v1-q4
cd trinity-v1-7b
touch model.json
```

The following is an example template for `model.json`
Copy the following into the `model.json`

```
```js
{
"source_url": "https://huggingface.co/janhq/trinity-v1-GGUF/resolve/main/trinity-v1.Q4_K_M.gguf",
"id": "trinity-v1-7b",
Expand All @@ -71,21 +73,25 @@ The following is an example template for `model.json`
"size": 4370000000
},
"engine": "nitro"
}
}
```

:::caution
Ensure the `source_url` is the link to download model
Ensure the `source_url` property is the direct binary download link ending in `.gguf`. Find the links in Huggingface > `Files and versions`.

Ensure the `id` is the same with the new created folder
Ensure the `id` property is the model foldername

Ensure to choose right `prompt_template`
Ensure you are using the correct `prompt_template`
:::

# 3. Use your model
## 3. Download your model binary

Restart the Jan application and look for your model in the Hub.

Click the green `download` button to download your actual model binary from the URL you provided.

![image](https://hackmd.io/_uploads/HJLAqvwI6.png)

There you go. If you have any questions or want to request for more preconfigured GGUF models, please message us on [Jan Discord](https://discord.gg/Dt7MxDyNNZ).
There you go! You are ready to use your model.

If you have any questions or want to request for more preconfigured GGUF models, please message us in [Discord](https://discord.gg/Dt7MxDyNNZ).

0 comments on commit 79f4bd1

Please sign in to comment.