Skip to content

Commit

Permalink
docs: add Jan installation using Docker
Browse files Browse the repository at this point in the history
  • Loading branch information
0xHieu01 committed Feb 9, 2024
1 parent eb09399 commit d518370
Show file tree
Hide file tree
Showing 4 changed files with 117 additions and 12 deletions.
27 changes: 15 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ To reset your installation:
- Clear Application cache in `~/Library/Caches/jan`

## Requirements for running Jan

- MacOS: 13 or higher
- Windows:
- Windows 10 or higher
Expand Down Expand Up @@ -194,17 +195,17 @@ Contributions are welcome! Please read the [CONTRIBUTING.md](CONTRIBUTING.md) fi

1. **Clone the repository and prepare:**

```bash
git clone https://github.com/janhq/jan
cd jan
git checkout -b DESIRED_BRANCH
```
```bash
git clone https://github.com/janhq/jan
cd jan
git checkout -b DESIRED_BRANCH
```

2. **Run development and use Jan Desktop**

```bash
make dev
```
```bash
make dev
```

This will start the development server and open the desktop app.

Expand All @@ -222,14 +223,15 @@ This will build the app MacOS m1/m2 for production (with code signing already do

- Supported OS: Linux, WSL2 Docker
- Pre-requisites:
- `docker` and `docker compose`, follow instruction [here](https://docs.docker.com/engine/install/ubuntu/)

- Docker Engine and Docker Compose are required to run Jan in Docker mode. Follow the [instructions](https://docs.docker.com/engine/install/ubuntu/) below to get started with Docker Engine on Ubuntu.

```bash
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh ./get-docker.sh --dry-run
```

- `nvidia-driver` and `nvidia-docker2`, follow instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) (If you want to run with GPU mode)
- If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation.

- Run Jan in Docker mode

Expand All @@ -241,7 +243,7 @@ This will build the app MacOS m1/m2 for production (with code signing already do

- **Option 2**: Run Jan in GPU mode

- **Step 1**: Check cuda compatibility with your nvidia driver by running `nvidia-smi` and check the cuda version in the output
- **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output

```bash
nvidia-smi
Expand Down Expand Up @@ -274,7 +276,7 @@ This will build the app MacOS m1/m2 for production (with code signing already do
|=======================================================================================|
```

- **Step 2**: Go to https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags and find the smallest minor version of image tag that matches the cuda version from the output of `nvidia-smi` (e.g. 12.1 -> 12.1.0)
- **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)

- **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`)

Expand All @@ -286,6 +288,7 @@ This will build the app MacOS m1/m2 for production (with code signing already do
```

This will start the web server and you can access Jan at `http://localhost:3000`.

> Note: Currently, Docker mode is only work for development and localhost, production is not supported yet. RAG feature is not supported in Docker mode yet.

## Acknowledgements
Expand Down
102 changes: 102 additions & 0 deletions docs/docs/guides/02-installation/05-docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
title: Docker
slug: /install/docker
description: Jan is a ChatGPT-alternative that runs on your own computer, with a local API server.
keywords:
[
Jan AI,
Jan,
ChatGPT alternative,
local AI,
private AI,
conversational AI,
no-subscription fee,
large language model,
docker installation,
]
---

# Installing Jan using Docker

## Installation

### Pre-requisites

:::note

**Supported OS**: Linux, WSL2 Docker

:::

- Docker Engine and Docker Compose are required to run Jan in Docker mode. Follow the [instructions](https://docs.docker.com/engine/install/ubuntu/) below to get started with Docker Engine on Ubuntu.

```bash
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh ./get-docker.sh --dry-run
```

- If you intend to run Jan in GPU mode, you need to install `nvidia-driver` and `nvidia-docker2`. Follow the instruction [here](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for installation.

### Instructions

- Run Jan in Docker mode

- **Option 1**: Run Jan in CPU mode

```bash
docker compose --profile cpu up -d
```

- **Option 2**: Run Jan in GPU mode

- **Step 1**: Check CUDA compatibility with your NVIDIA driver by running `nvidia-smi` and check the CUDA version in the output

```bash
nvidia-smi
# Output
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A |
| 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A |
| 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A |
| 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
```

- **Step 2**: Visit [NVIDIA NGC Catalog ](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags) and find the smallest minor version of image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)

- **Step 3**: Update the `Dockerfile.gpu` line number 5 with the latest minor version of the image tag from step 2 (e.g. change `FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base` to `FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base`)

- **Step 4**: Run command to start Jan in GPU mode

```bash
# GPU mode
docker compose --profile gpu up -d
```

This will start the web server and you can access Jan at `http://localhost:3000`.

:::warning

- Docker mode is currently only suitable for development and localhost. Production is not supported yet, and the RAG feature is not available in Docker mode.

:::

0 comments on commit d518370

Please sign in to comment.