Bump nitro to 0.2.8 and change Jan App to support cuda >= 11.7 (janhq…

…#1476)

.github/workflows/jan-electron-linter-and-test.yml

@@ -28,6 +28,9 @@ on:
       - "node_modules/**"
       - "yarn.lock"
       - "Makefile"
+      - "extensions/**"
+      - "core/**"
+      - "!README.md"
 
 jobs:
   test-on-macos:

README.md

@@ -173,7 +173,7 @@ To reset your installation:
   - To enable GPU support:
     - Nvidia GPU with CUDA Toolkit 11.7 or higher
     - Nvidia driver 470.63.01 or higher
-Linux:
+- Linux:
   - glibc 2.27 or higher (check with `ldd --version`)
   - gcc 11, g++ 11, cpp 11 or higher, refer to this [link](https://jan.ai/guides/troubleshooting/gpu-not-used/#specific-requirements-for-linux) for more information
   - To enable GPU support:

docs/docs/guides/08-troubleshooting/03-gpu-not-used.mdx

@@ -21,7 +21,7 @@ This guide provides steps to troubleshoot and resolve issues when Jan app does n
 
 ### NVIDIA Driver
 
-Ensure that you have installed the NVIDIA driver that supports CUDA 11.4 or higher. For a detailed of CUDA compatibility, please refer [here](https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver).
+Ensure that you have installed the NVIDIA driver that supports CUDA 11.7 or higher. For a detailed of CUDA compatibility, please refer [here](https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver).
 
 To verify, open PowerShell or Terminal and enter the following command:
 
@@ -33,7 +33,7 @@ If you see a result similar to the following, you have successfully installed th
 
 ```bash
 +-----------------------------------------------------------------------------+
-| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.4     |
+| NVIDIA-SMI 470.57.02    Driver Version: 470.57.02    CUDA Version: 11.7     |
 |-------------------------------+----------------------+----------------------+
 | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
 | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
@@ -60,8 +60,8 @@ If you see a result similar to the following, you have successfully installed CU
 ```bash
 nvcc: NVIDIA (R) Cuda compiler driver
 
-Cuda compilation tools, release 11.4, V11.4.100
-Build cuda_11.4.r11.4/compiler.30033411_0
+Cuda compilation tools, release 11.7, V11.7.100
+Build cuda_11.7.r11.7/compiler.30033411_0
 ```
 
 ### Specific Requirements for Linux
@@ -91,11 +91,11 @@ g++ --version
 **Post-Installation Actions**: You must add the `.so` libraries of CUDA to the `LD_LIBRARY_PATH` environment variable by following the [Post-installation Actions instruction](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions).
 
 ```bash
-# Example for ubuntu with CUDA 11.4
+# Example for ubuntu with CUDA 11.7
 sudo nano /etc/environment
-# Add /usr/local/cuda-11.4/bin to the PATH environment variable - the first line
+# Add /usr/local/cuda-11.7/bin to the PATH environment variable - the first line
 # Add the following line to the end of the file
-LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64
+LD_LIBRARY_PATH=/usr/local/cuda-11.7/lib64
 
 # Save and exit
 # Restart your computer or log out and log in again, the changes will take effect
@@ -111,7 +111,7 @@ If you find that GPU mode is available but not enabled by default, consider the
 
 :::tip
 
-1. Check if you have installed the NVIDIA driver that supports CUDA 11.4 or higher. For a detailed of CUDA compatibility, please refer [here](https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver).
+1. Check if you have installed the NVIDIA driver that supports CUDA 11.7 or higher. For a detailed of CUDA compatibility, please refer [here](https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver).
 
 2. Ensure that the CUDA toolkit is installed and compatible with your NVIDIA driver. For a detailed of CUDA compatibility, please refer [here](https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver).
 

extensions/inference-nitro-extension/bin/version.txt

@@ -1 +1 @@
-0.2.7
+0.2.8

extensions/inference-nitro-extension/download.bat

@@ -1,3 +1,3 @@
 @echo off
 set /p NITRO_VERSION=<./bin/version.txt
-.\node_modules\.bin\download https://github.com/janhq/nitro/releases/download/v%NITRO_VERSION%/nitro-%NITRO_VERSION%-win-amd64-cuda-12-0.tar.gz -e --strip 1 -o ./bin/win-cuda-12-0 && .\node_modules\.bin\download https://github.com/janhq/nitro/releases/download/v%NITRO_VERSION%/nitro-%NITRO_VERSION%-win-amd64-cuda-11-4.tar.gz -e --strip 1 -o ./bin/win-cuda-11-4 && .\node_modules\.bin\download https://github.com/janhq/nitro/releases/download/v%NITRO_VERSION%/nitro-%NITRO_VERSION%-win-amd64.tar.gz -e --strip 1 -o ./bin/win-cpu
+.\node_modules\.bin\download https://github.com/janhq/nitro/releases/download/v%NITRO_VERSION%/nitro-%NITRO_VERSION%-win-amd64-cuda-12-0.tar.gz -e --strip 1 -o ./bin/win-cuda-12-0 && .\node_modules\.bin\download https://github.com/janhq/nitro/releases/download/v%NITRO_VERSION%/nitro-%NITRO_VERSION%-win-amd64-cuda-11-7.tar.gz -e --strip 1 -o ./bin/win-cuda-11-7 && .\node_modules\.bin\download https://github.com/janhq/nitro/releases/download/v%NITRO_VERSION%/nitro-%NITRO_VERSION%-win-amd64.tar.gz -e --strip 1 -o ./bin/win-cpu

extensions/inference-nitro-extension/package.json

@@ -8,7 +8,7 @@
   "license": "AGPL-3.0",
   "scripts": {
     "build": "tsc -b . && webpack --config webpack.config.js",
-    "downloadnitro:linux": "NITRO_VERSION=$(cat ./bin/version.txt) && download https://github.com/janhq/nitro/releases/download/v${NITRO_VERSION}/nitro-${NITRO_VERSION}-linux-amd64.tar.gz -e --strip 1 -o ./bin/linux-cpu && chmod +x ./bin/linux-cpu/nitro && download https://github.com/janhq/nitro/releases/download/v${NITRO_VERSION}/nitro-${NITRO_VERSION}-linux-amd64-cuda-12-0.tar.gz -e --strip 1 -o ./bin/linux-cuda-12-0 && chmod +x ./bin/linux-cuda-12-0/nitro && download https://github.com/janhq/nitro/releases/download/v${NITRO_VERSION}/nitro-${NITRO_VERSION}-linux-amd64-cuda-11-4.tar.gz -e --strip 1 -o ./bin/linux-cuda-11-4 && chmod +x ./bin/linux-cuda-11-4/nitro",
+    "downloadnitro:linux": "NITRO_VERSION=$(cat ./bin/version.txt) && download https://github.com/janhq/nitro/releases/download/v${NITRO_VERSION}/nitro-${NITRO_VERSION}-linux-amd64.tar.gz -e --strip 1 -o ./bin/linux-cpu && chmod +x ./bin/linux-cpu/nitro && download https://github.com/janhq/nitro/releases/download/v${NITRO_VERSION}/nitro-${NITRO_VERSION}-linux-amd64-cuda-12-0.tar.gz -e --strip 1 -o ./bin/linux-cuda-12-0 && chmod +x ./bin/linux-cuda-12-0/nitro && download https://github.com/janhq/nitro/releases/download/v${NITRO_VERSION}/nitro-${NITRO_VERSION}-linux-amd64-cuda-11-7.tar.gz -e --strip 1 -o ./bin/linux-cuda-11-7 && chmod +x ./bin/linux-cuda-11-7/nitro",
     "downloadnitro:darwin": "NITRO_VERSION=$(cat ./bin/version.txt) && download https://github.com/janhq/nitro/releases/download/v${NITRO_VERSION}/nitro-${NITRO_VERSION}-mac-arm64.tar.gz -e --strip 1 -o ./bin/mac-arm64 && chmod +x ./bin/mac-arm64/nitro && download https://github.com/janhq/nitro/releases/download/v${NITRO_VERSION}/nitro-${NITRO_VERSION}-mac-amd64.tar.gz -e --strip 1 -o ./bin/mac-x64 && chmod +x ./bin/mac-x64/nitro",
     "downloadnitro:win32": "download.bat",
     "downloadnitro": "run-script-os",

extensions/inference-nitro-extension/src/module.ts

@@ -395,7 +395,7 @@ function spawnNitroProcess(nitroResourceProbe: any): Promise<any> {
         if (nvidiaInfo["cuda"].version === "12") {
           binaryFolder = path.join(binaryFolder, "win-cuda-12-0");
         } else {
-          binaryFolder = path.join(binaryFolder, "win-cuda-11-4");
+          binaryFolder = path.join(binaryFolder, "win-cuda-11-7");
         }
         cudaVisibleDevices = nvidiaInfo["gpu_highest_vram"];
       }
@@ -415,7 +415,7 @@ function spawnNitroProcess(nitroResourceProbe: any): Promise<any> {
         if (nvidiaInfo["cuda"].version === "12") {
           binaryFolder = path.join(binaryFolder, "linux-cuda-12-0");
         } else {
-          binaryFolder = path.join(binaryFolder, "linux-cuda-11-4");
+          binaryFolder = path.join(binaryFolder, "linux-cuda-11-7");
         }
         cudaVisibleDevices = nvidiaInfo["gpu_highest_vram"];
       }