feat: integrating cortex (janhq#3001)

* feat: integrating cortex * Temporary prevent crash Signed-off-by: James <[email protected]> * fix yarn lint Signed-off-by: James <[email protected]> * refactor: remove core node module - fs - extensions and so on (janhq#3151) * add migration script for threads, messages and models Signed-off-by: James <[email protected]> * remove freq_penalty and presence_penalty if model not supported Signed-off-by: James <[email protected]> * add back models in my models Signed-off-by: James <[email protected]> * fix api-url for setup API key popup Signed-off-by: James <[email protected]> * fix using model name for dropdown model Signed-off-by: James <[email protected]> * fix can't click to hotkey Signed-off-by: James <[email protected]> * fix: disable some UIs Signed-off-by: James <[email protected]> * fix build Signed-off-by: James <[email protected]> * reduce calling HF api Signed-off-by: James <[email protected]> * some ui update Signed-off-by: James <[email protected]> * feat: modal migration UI (janhq#3153) * feat: handle popup migration * chore: update loader * chore: integrate script migration * chore: cleanup import * chore: moving out spinner loader * chore: update check thread message success migrate * chore: add handle script into retry button * remove warning from joi Signed-off-by: James <[email protected]> * chore: fix duplicate children * fix: path after migrating model Signed-off-by: James <[email protected]> * chore: apply mutation for config * chore: prevent calling too many create assistant api Signed-off-by: James <[email protected]> * using cortexso Signed-off-by: James <[email protected]> * update download api Signed-off-by: James <[email protected]> * fix use on slider item Signed-off-by: James <[email protected]> * fix: ui no download model or simple onboarding (janhq#3166) * fix download huggingface model match with slider item Signed-off-by: James <[email protected]> * update owner_logo to logo and author Signed-off-by: James <[email protected]> * update new cortexso Signed-off-by: James <[email protected]> * Add install python step for macos * add engine table Signed-off-by: James <[email protected]> * fix local icons Signed-off-by: James <[email protected]> * feat: add search feature for model hub Signed-off-by: James <[email protected]> * fix misalign switch Signed-off-by: James <[email protected]> * fix: delete thread not focus on other thread Signed-off-by: James <[email protected]> * add get model from hugging face Signed-off-by: James <[email protected]> * fix download from hugging face Signed-off-by: James <[email protected]> * small update Signed-off-by: James <[email protected]> * update Signed-off-by: James <[email protected]> * fix system monitor rounded only on the left Signed-off-by: James <[email protected]> * chore: update ui new hub screen (janhq#3174) * chore: update ui new hub screen * chore: update layout centerpanel thread and hub screen * chore: update detail model by group * update cortexso 0.1.13 Signed-off-by: James <[email protected]> * chore: add file size Signed-off-by: James <[email protected]> * chore: put engine to experimental feature Signed-off-by: James <[email protected]> * chore: open cortex folder Signed-off-by: James <[email protected]> * chore: add back user avatar Signed-off-by: James <[email protected]> * chore: minor UI hub (janhq#3182) * chore: add back right click thread list and update 3 dots are overlapping with the text * chore: update position dropdown list my models * chore: make on-device tab showing 6 items instead of 4 * chore: update style description modals detail model * chore: update isGeneration loader and author name on modal * feat: integrate cortex single executable Signed-off-by: James <[email protected]> * fix build Signed-off-by: James <[email protected]> * chore: added blank state * chore: update ui component blank state * bump cortex binary version * fix: logic show modal migration (janhq#3165) * fix: logic show modal migration * chore: fixed logic * chore: read contain format gguf local models * chore: change return hasLocalModel * chore: intiial skipmigration state * chore: filter embedding model * fix: delete top thread not focus on any other thread * chore: added UI no result component search models group (janhq#3188) * fix: remote model should show all when user config that engine Signed-off-by: James <[email protected]> * chore: set state thread and models migration using getOnInit (janhq#3189) * chore: set state thread and models migration using getOnInit * chore: add state as dependecies hooks * chore: system monitor panel show engine model (janhq#3192) * fix: remove config api, replace with engine Signed-off-by: James <[email protected]> * update Signed-off-by: James <[email protected]> * update reactquery Signed-off-by: James <[email protected]> * bump cortex 0.4.35 * feat: add waiting for cortex popup Signed-off-by: James <[email protected]> * chore: add loader detail model popup (janhq#3195) * chore: model start loader (janhq#3197) * chore: added model loader when user starting chat without model active * chore: update copies loader * fix: select min file size if recommended quant does not exist Signed-off-by: James <[email protected]> * chore: temporary hide gpu config * fix: tensorrt not shown Signed-off-by: James <[email protected]> * fix lint Signed-off-by: James <[email protected]> * fix tests Signed-off-by: James <[email protected]> * fix e2e tests (wip) Signed-off-by: James <[email protected]> * update Signed-off-by: James <[email protected]> * fix: adding element and correct test to adapt new UI * fix: temp skip unstable part * fix: only show models which can be supported Signed-off-by: James <[email protected]> * Update version.txt * update send message Signed-off-by: James <[email protected]> * fix: not allow user send message when is generating Signed-off-by: James <[email protected]> * chore: temp skip Playwright test due to env issue * chore: temp skip Playwright test due to env issue * update Signed-off-by: James <[email protected]> * chore: minor-ui-feedback (janhq#3202) --------- Signed-off-by: James <[email protected]> Co-authored-by: Louis <[email protected]> Co-authored-by: Faisal Amir <[email protected]> Co-authored-by: Hien To <[email protected]> Co-authored-by: Van Pham <[email protected]> Co-authored-by: Van-QA <[email protected]>
satendrakumar · Jul 26, 2024 · 101268f · 101268f
1 parent 7a660ad
commit 101268f
Show file tree

Hide file tree

Showing 536 changed files with 20,672 additions and 28,852 deletions.
diff --git a/.github/workflows/jan-electron-linter-and-test.yml b/.github/workflows/jan-electron-linter-and-test.yml
@@ -67,9 +67,9 @@ jobs:
         run: |
           echo "REPORT_PORTAL_DESCRIPTION=${{github.sha}})" >> $GITHUB_ENV
 
-      - name: 'Config report portal'
-        run: |
-          make update-playwright-config REPORT_PORTAL_URL=${{ secrets.REPORT_PORTAL_URL }} REPORT_PORTAL_API_KEY=${{ secrets.REPORT_PORTAL_API_KEY }} REPORT_PORTAL_PROJECT_NAME=${{ secrets.REPORT_PORTAL_PROJECT_NAME }} REPORT_PORTAL_LAUNCH_NAME="Jan App macos" REPORT_PORTAL_DESCRIPTION="${{env.REPORT_PORTAL_DESCRIPTION}}"
+#      - name: 'Config report portal'
+#        run: |
+#          make update-playwright-config REPORT_PORTAL_URL=${{ secrets.REPORT_PORTAL_URL }} REPORT_PORTAL_API_KEY=${{ secrets.REPORT_PORTAL_API_KEY }} REPORT_PORTAL_PROJECT_NAME=${{ secrets.REPORT_PORTAL_PROJECT_NAME }} REPORT_PORTAL_LAUNCH_NAME="Jan App macos" REPORT_PORTAL_DESCRIPTION="${{env.REPORT_PORTAL_DESCRIPTION}}"
 
       - name: Linter and test
         run: |
@@ -147,10 +147,10 @@ jobs:
         run: |
           echo "REPORT_PORTAL_DESCRIPTION=${{github.sha}}" >> $GITHUB_ENV
 
-      - name: 'Config report portal'
-        shell: bash
-        run: |
-          make update-playwright-config REPORT_PORTAL_URL=${{ secrets.REPORT_PORTAL_URL }} REPORT_PORTAL_API_KEY=${{ secrets.REPORT_PORTAL_API_KEY }} REPORT_PORTAL_PROJECT_NAME=${{ secrets.REPORT_PORTAL_PROJECT_NAME }} REPORT_PORTAL_LAUNCH_NAME="Jan App Windows ${{ matrix.antivirus-tools }}" REPORT_PORTAL_DESCRIPTION="${{env.REPORT_PORTAL_DESCRIPTION}}"
+#      - name: 'Config report portal'
+#        shell: bash
+#        run: |
+#          make update-playwright-config REPORT_PORTAL_URL=${{ secrets.REPORT_PORTAL_URL }} REPORT_PORTAL_API_KEY=${{ secrets.REPORT_PORTAL_API_KEY }} REPORT_PORTAL_PROJECT_NAME=${{ secrets.REPORT_PORTAL_PROJECT_NAME }} REPORT_PORTAL_LAUNCH_NAME="Jan App Windows ${{ matrix.antivirus-tools }}" REPORT_PORTAL_DESCRIPTION="${{env.REPORT_PORTAL_DESCRIPTION}}"
 
       - name: Linter and test
         shell: powershell
@@ -195,10 +195,10 @@ jobs:
         run: |
           echo "REPORT_PORTAL_DESCRIPTION=${{github.event.after}}" >> $GITHUB_ENV
 
-      - name: 'Config report portal'
-        shell: bash
-        run: |
-          make update-playwright-config REPORT_PORTAL_URL=${{ secrets.REPORT_PORTAL_URL }} REPORT_PORTAL_API_KEY=${{ secrets.REPORT_PORTAL_API_KEY }} REPORT_PORTAL_PROJECT_NAME=${{ secrets.REPORT_PORTAL_PROJECT_NAME }} REPORT_PORTAL_LAUNCH_NAME="Jan App Windows" REPORT_PORTAL_DESCRIPTION="${{env.REPORT_PORTAL_DESCRIPTION}}"
+#      - name: 'Config report portal'
+#        shell: bash
+#        run: |
+#          make update-playwright-config REPORT_PORTAL_URL=${{ secrets.REPORT_PORTAL_URL }} REPORT_PORTAL_API_KEY=${{ secrets.REPORT_PORTAL_API_KEY }} REPORT_PORTAL_PROJECT_NAME=${{ secrets.REPORT_PORTAL_PROJECT_NAME }} REPORT_PORTAL_LAUNCH_NAME="Jan App Windows" REPORT_PORTAL_DESCRIPTION="${{env.REPORT_PORTAL_DESCRIPTION}}"
 
       - name: Linter and test
         shell: powershell
@@ -275,10 +275,10 @@ jobs:
         run: |
           echo "REPORT_PORTAL_DESCRIPTION=${{github.sha}}" >> $GITHUB_ENV
 
-      - name: 'Config report portal'
-        shell: bash
-        run: |
-          make update-playwright-config REPORT_PORTAL_URL=${{ secrets.REPORT_PORTAL_URL }} REPORT_PORTAL_API_KEY=${{ secrets.REPORT_PORTAL_API_KEY }} REPORT_PORTAL_PROJECT_NAME=${{ secrets.REPORT_PORTAL_PROJECT_NAME }} REPORT_PORTAL_LAUNCH_NAME="Jan App Linux" REPORT_PORTAL_DESCRIPTION="${{env.REPORT_PORTAL_DESCRIPTION}}"
+#      - name: 'Config report portal'
+#        shell: bash
+#        run: |
+#          make update-playwright-config REPORT_PORTAL_URL=${{ secrets.REPORT_PORTAL_URL }} REPORT_PORTAL_API_KEY=${{ secrets.REPORT_PORTAL_API_KEY }} REPORT_PORTAL_PROJECT_NAME=${{ secrets.REPORT_PORTAL_PROJECT_NAME }} REPORT_PORTAL_LAUNCH_NAME="Jan App Linux" REPORT_PORTAL_DESCRIPTION="${{env.REPORT_PORTAL_DESCRIPTION}}"
 
       - name: Linter and test
         run: |

diff --git a/.github/workflows/template-build-macos-arm64.yml b/.github/workflows/template-build-macos-arm64.yml
@@ -56,6 +56,11 @@ jobs:
         with:
           node-version: 20
 
+      - name: Install python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.9'
+
       - name: Install jq
         uses: dcarbone/[email protected]
 

diff --git a/.github/workflows/template-build-macos-x64.yml b/.github/workflows/template-build-macos-x64.yml
@@ -56,6 +56,11 @@ jobs:
         with:
           node-version: 20
 
+      - name: Install python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.9'
+
       - name: Install jq
         uses: dcarbone/[email protected]
 

diff --git a/.gitignore b/.gitignore
@@ -12,6 +12,9 @@ yarn.lock
 dist
 build
 .DS_Store
+electron/resources/win/*
+electron/resources/linux/*
+electron/resources/mac/*
 electron/renderer
 electron/models
 electron/docs

diff --git a/Dockerfile b/Dockerfile
@@ -32,8 +32,6 @@ COPY --from=builder /app/yarn.lock ./yarn.lock
 COPY --from=builder /app/core ./core/
 COPY --from=builder /app/server ./server/
 RUN cd core && yarn install && yarn run build
-RUN yarn workspace @janhq/server install && yarn workspace @janhq/server build
-COPY --from=builder /app/docs/openapi ./docs/openapi/
 
 # Copy pre-install dependencies
 COPY --from=builder /app/pre-install ./pre-install/

diff --git a/Dockerfile.gpu b/Dockerfile.gpu
@@ -56,8 +56,6 @@ COPY --from=builder /app/yarn.lock ./yarn.lock
 COPY --from=builder /app/core ./core/
 COPY --from=builder /app/server ./server/
 RUN cd core && yarn install && yarn run build
-RUN yarn workspace @janhq/server install && yarn workspace @janhq/server build
-COPY --from=builder /app/docs/openapi ./docs/openapi/
 
 # Copy pre-install dependencies
 COPY --from=builder /app/pre-install ./pre-install/

diff --git a/Makefile b/Makefile
@@ -18,16 +18,14 @@ else
 	cd joi && yarn install && yarn build
 endif
 
-# Installs yarn dependencies and builds core and extensions
+# Installs yarn dependencies and builds core
 install-and-build: build-joi
 ifeq ($(OS),Windows_NT)
 	yarn config set network-timeout 300000
 endif
 	yarn global add [email protected]
 	yarn build:core
-	yarn build:server
 	yarn install
-	yarn build:extensions
 
 check-file-counts: install-and-build
 ifeq ($(OS),Windows_NT)

diff --git a/README.md b/README.md
@@ -210,12 +210,6 @@ Contributions are welcome! Please read the [CONTRIBUTING.md](CONTRIBUTING.md) fi
 
 This will start the development server and open the desktop app.
 
-3. (Optional) **Run the API server without frontend**
-
-   ```bash
-   yarn dev:server
-   ```
-
 ### For production build
 
 ```bash

diff --git a/core/README.md b/core/README.md
@@ -40,7 +40,7 @@ import * as node from "@janhq/core/node";
         private static inference(incomingMessage: MessageRequestData) {
 
           // Prepare customized message content
-          const content: ThreadContent = {
+          const content: MessageContent = {
             type: ContentType.Text,
             text: {
               value: "I'm Jan Assistant!",
@@ -49,7 +49,7 @@ import * as node from "@janhq/core/node";
           };
 
           // Modify message and send out
-          const outGoingMessage: ThreadMessage = {
+          const outGoingMessage: Message = {
             ...incomingMessage,
             content
           };

diff --git a/core/package.json b/core/package.json
@@ -8,7 +8,7 @@
   ],
   "homepage": "https://jan.ai",
   "license": "AGPL-3.0",
-  "main": "dist/core.es5.js",
+  "main": "dist/lib/index.js",
   "module": "dist/core.cjs.js",
   "typings": "dist/types/index.d.ts",
   "files": [
@@ -17,18 +17,18 @@
   ],
   "author": "Jan <[email protected]>",
   "exports": {
-    ".": "./dist/core.es5.js",
+    ".": "./dist/lib/index.js",
     "./node": "./dist/node/index.cjs.js"
   },
   "typesVersions": {
     "*": {
       ".": [
-        "./dist/core.es5.js.map",
+        "./dist/lib/index.js",
         "./dist/types/index.d.ts"
       ],
       "node": [
         "./dist/node/index.cjs.js.map",
-        "./dist/types/node/index.d.ts"
+        "./dist/types/index.d.ts"
       ]
     }
   },
@@ -40,6 +40,7 @@
     "start": "rollup -c rollup.config.ts -w"
   },
   "devDependencies": {
+    "openai": "4.51.0",
     "@rollup/plugin-replace": "^5.0.5",
     "@types/jest": "^29.5.12",
     "@types/node": "^20.11.4",
@@ -58,7 +59,6 @@
     "typescript": "^5.3.3"
   },
   "dependencies": {
-    "rxjs": "^7.8.1",
-    "ulidx": "^2.3.0"
+    "rxjs": "^7.8.1"
   }
 }
diff --git a/core/rollup.config.ts b/core/rollup.config.ts
@@ -43,7 +43,7 @@ export default [
     ],
   },
   {
-    input: `src/node/index.ts`,
+    input: `src/index.ts`,
     output: [{ file: 'dist/node/index.cjs.js', format: 'cjs', sourcemap: true }],
     // Indicate here external modules you don't wanna include in your bundle (i.e.: 'lodash')
     external: [
@@ -52,7 +52,6 @@ export default [
       'pacote',
       '@types/pacote',
       '@npmcli/arborist',
-      'ulidx',
       'node-fetch',
       'fs',
       'request',

diff --git a/core/src/browser/core.ts b/core/src/browser/core.ts