Skip to content

Commit

Permalink
fixes for the multi-model-exec demo
Browse files Browse the repository at this point in the history
- updates to CMakelist compile issue for OPENCV_DIR
  • Loading branch information
cyndwith committed Aug 28, 2024
1 parent 631cb1a commit c301451
Show file tree
Hide file tree
Showing 166 changed files with 3,478 additions and 26,789 deletions.
Binary file removed demo/multi-model-exec/1x4.xclbin
Binary file not shown.
121 changes: 61 additions & 60 deletions demo/multi-model-exec/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@
- [1 Introduction](#1-Introduction)
- [2 Design Files](#2-Design-Files)
- [3 Demo Setup](#3-Demo-Setup)
- [3.1 Environment Setup](#31-Environment-Setup)
- [3.2 Demo Preparation](#32-Demo-Preparation)
- [4 Run The Demo](#4-Run-The-Demo)
- [5 Know Issues](#5-Know-Issues)

Expand All @@ -34,19 +32,16 @@ This is the demo of showing multiple AI models running concurrently on Ryzen AI.


```
│ 1x4.xclbin
│ env.yaml
│ README.md
│ vaip_config.json
│ voe-0.1.0-cp39-cp39-win_amd64.whl
├─.vscode
│ settings.json
├─bin
ipu_multi_models.exe
npu_multi_models.exe
│ onnxruntime.dll
│ onnxruntime_providers_shared.dll
│ onnxruntime_providers_vitisai.dll
│ onnxruntime_vitisai_ep.dll
│ DirectML.dll
│ vaip_config.json
├─images
│ mobilenet_V2.jpg
Expand All @@ -56,12 +51,19 @@ This is the demo of showing multiple AI models running concurrently on Ryzen AI.
│ segmentation.jpg
│ yolox.jpg
└─ipu_modelsx4_demo
│ generate_script.py
└─npu_modelsx4_demo
│ run_mobile_net_v2.bat
│ run_modelx4.bat
│ run_modelx4_with_camera_on.bat
│ run_resnet50.bat
│ run_retinaface.bat
│ run_segmentation.bat
│ run_yolovx.bat
└─config
mobile_net_v2.json
modelx4.json
modelx4_with_camera_on.json
resnet50.json
retinaface.json
segmentation.json
Expand All @@ -70,59 +72,58 @@ This is the demo of showing multiple AI models running concurrently on Ryzen AI.

## 3 Demo Setup

### 3.1 Environment Setup

Please note that all the steps mentioned here need to be performed in ***Windows CMD Prompt***. Make sure the IPU driver has been installed by following the instructions on the [Ryzen AI Developer Guide](https://ryzenai.docs.amd.com/en/latest). The IPU driver version >= 10.106.8.62 has been tested. The older version might work but no guaranteed.

1. Follow these steps to install Anaconda on your system if it has not been installed:

- **Download Anaconda:** Visit the Anaconda download page (https://www.anaconda.com/products/distribution) and choose the distribution that matches your operating system (Windows, macOS, or Linux). Download the Python 3.9 version.

- **Install Anaconda:** Follow the installation instructions for your operating system. Generally, this involves running the installer and accepting the default settings.

- **Verify Installation:** After installation, open Anaconda Prompt on Windows and type `conda --version` to verify that Anaconda was installed correctly. You should see the installed Conda version.
Make sure you have met all the requirements by following the [Installation Instructions](https://ryzenai.docs.amd.com/en/latest/inst.html#).

2. Create a Conda Environment by using the env.yaml
```bash
conda create --name <env_name> --file=env.yaml
```
Download the onnx models and test image/video package, and unzip it under **<path_to_RyzenAI-SW>/demo/multi-model-exec/npu_modelsx4_demo/**

3. Activate the Environment
```bash
conda activate <env_name>
```
### 3.2 Demo Preparation

1. Download the onnx models and test image/video package, and unzip it under **<path_to_RyzenAI-SW>/demo/multi-model-exec/ipu_modelsx4_demo/**.


[Download Package](https://www.xilinx.com/bin/public/openDownload?filename=resource_multi_model_demo.zip)
[Download Resource Package](https://www.xilinx.com/bin/public/openDownload?filename=resource_multi_model_demo.zip)

You should have the directory like this:
```bash
.
├── generate_script.py
├── modelsx4_screenshot.png
├── resource
│   ├── RetinaFace_int.onnx
│   ├── detection
│   ├── detection.avi
│   ├── face
│   ├── face.avi
│   ├── mobilenetv2_1.4_int.onnx
│   ├── nano-YOLOX_int.onnx
│   ├── pointpainting-nus-FPN_int.onnx
│   ├── resnet50_pt.onnx
│   ├── seg_512_288.avi
│   └── to_video.py
```

2. Generate the config files by running the comand:

```bash
python generate_script.py
```
├── bin
│   ├── DirectML.dll
│   ├── npu_multi_models.exe
│   ├── onnxruntime.dll
│   ├── onnxruntime_providers_shared.dll
│   ├── onnxruntime_providers_vitisai.dll
│   ├── onnxruntime_vitisai_ep.dll
│   └── vaip_config.json
├── images
│   ├── mobilenet_V2.jpg
│   ├── modelsx4.jpg
│   ├── resnet50.jpg
│   ├── retina.jpg
│   ├── segmentation.jpg
│   └── yolox.jpg
├── npu_modelsx4_demo
│   ├── config
│   │   ├── mobile_net_v2.json
│   │   ├── modelx4.json
│   │   ├── modelx4_with_camera_on.json
│   │   ├── resnet50.json
│   │   ├── retinaface.json
│   │   ├── segmentation.json
│   │   └── yolovx.json
│   ├── resource
│   │   ├── detection
│   │   ├── face
│   │   ├── mobilenetv2_1.4_int.onnx
│   │   ├── nano-YOLOX_int.onnx
│   │   ├── pointpainting-nus-FPN_int.onnx
│   │   ├── resnet50_pt.onnx
│   │   ├── RetinaFace_int.onnx
│   │   └── seg_512_288.avi
│   ├── run_mobile_net_v2.bat
│   ├── run_modelx4.bat
│   ├── run_modelx4_with_camera_on.bat
│   ├── run_resnet50.bat
│   ├── run_retinaface.bat
│   ├── run_segmentation.bat
│   └── run_yolovx.bat
└── README.md

```


## 4 Run The Demo
Expand All @@ -149,6 +150,6 @@ run_modelx4.bat

## 5 Know Issues

- Python version version 3.9 is required if not "The code execution cannot proceed because python39.dll was not found. Reinstalling he programm may fix this problem"
- If you find an exclamation mark on the icon of the AMD IPU device in the System Devices list in your Device Manager, it indicates that there is an issue with your driver installation, and the program may not function correctly.
- Python version version 3.10 is required if not "The code execution cannot proceed because python39.dll was not found. Reinstalling he programm may fix this problem"
- If you find an exclamation mark on the icon of the AMD NPU device in the System Devices list in your Device Manager, it indicates that there is an issue with your driver installation, and the program may not function correctly.
- If this demo aborted with the 'glog.dll cannot be found' error, you need to use the command 'set PATH=C:<path-to-conda-glog>;%PATH%' to explicitly export the path to 'glog.dll'. 'glog.dll' is installed along with ANACONDA3. The recommended ANACONDA3 installer is 'Anaconda3-2023.07-2-Windows-x86_64'.
3 changes: 3 additions & 0 deletions demo/multi-model-exec/bin/DirectML.dll
Git LFS file not shown
Binary file removed demo/multi-model-exec/bin/ipu_multi_models.exe
Binary file not shown.
Binary file added demo/multi-model-exec/bin/npu_multi_models.exe
Binary file not shown.
4 changes: 2 additions & 2 deletions demo/multi-model-exec/bin/onnxruntime.dll
Git LFS file not shown
3 changes: 3 additions & 0 deletions demo/multi-model-exec/bin/onnxruntime_providers_shared.dll
Git LFS file not shown
3 changes: 3 additions & 0 deletions demo/multi-model-exec/bin/onnxruntime_providers_vitisai.dll
Git LFS file not shown
4 changes: 2 additions & 2 deletions demo/multi-model-exec/bin/onnxruntime_vitisai_ep.dll
Git LFS file not shown
173 changes: 173 additions & 0 deletions demo/multi-model-exec/bin/vaip_config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,173 @@
{
"passes": [
{
"name": "init",
"plugin": "vaip-pass_init"
},
{
"name": "fuse_DPU",
"plugin": "vaip-pass_level1_dpu",
"passDpuParam": {
"subPass": [
{
"name": "convert_ending_blacklist_ops_to_unknown_op",
"plugin": "vaip-pass_convert_ending_blacklist_ops_to_unknown_op",
"disabled": false
},
{
"name": "dynamic_input_batch",
"plugin": "vaip-pass_dynamic_input_batch"
},
{
"name": "create_const_op",
"plugin": "vaip-pass_create_const_op"
},
{
"name": "convert_split_to_xir",
"plugin": "vaip-pass_convert_split_to_xir_op"
},
{
"name": "to_xir",
"plugin": "vaip-pass_to_xir_ops"
},
{
"name": "convert_pad",
"plugin": "vaip-pass_convert_pad",
"enableGc": true
},
{
"name": "convert_in_to_gn",
"plugin": "vaip-pass_convert_instancenorm_to_groupnorm",
"enableGc": true
},
{
"name": "remove_extra_q_dq",
"plugin": "vaip-pass_remove_extra_q_dq"
},
{
"name": "merge_add_into_conv_bias",
"plugin": "vaip-pass_merge_add_into_conv_bias"
},
{
"name": "merge_fix",
"plugin": "vaip-pass_merge_fix",
"enableGc": true
},
{
"name": "layoutransform",
"plugin": "vaip-pass_layout_transform_via_adding_transpose"
},
{
"name": "gc_after_layout_transform",
"plugin": "vaip-pass_remove_isolated_node"
},
{
"name": "fuse_transpose",
"plugin": "vaip-pass_fuse_transpose",
"enableGc": true
},
{
"name": "gc_after_fuse_transpose",
"plugin": "vaip-pass_remove_isolated_node"
},
{
"name": "remove_identity",
"plugin": "vaip-pass_remove_identity",
"logVerbosity": 1
},
{
"name": "add_fix_after_const",
"plugin": "vaip-pass_const_add_fix"
},
{
"name": "remove_reshape_fix",
"plugin": "vaip-pass_py_ext",
"disabled": true,
"pyExt": {
"moduleName": "voe.passes.remove_reshape_fix",
"methodName": "rules"
}
},
{
"name": "const_fold_batchnorm_to_scale",
"plugin": "vaip-pass_const_fold_batchnorm_to_scale"
},
{
"name": "const_fold_transpose",
"plugin": "vaip-pass_const_fold_transpose"
},
{
"name": "merge_pad",
"plugin": "vaip-pass_merge_pad"
},
{
"name": "merge_hard_sigmoid",
"plugin": "vaip-pass_merge_hard_sigmoid"
},
{
"name": "merge_mul",
"plugin": "vaip-pass_merge_mul",
"enableGc": true
},
{
"name": "merge_consecutive_fix",
"plugin": "vaip-pass_merge_consecutive_fix",
"disabled": true,
"enableLog": true,
"logVerbosity": 1
},
{
"_comment": "test case 110",
"name": "convert_softmax_to_hard_softmax",
"plugin": "vaip-pass_convert_softmax_to_hard_softmax",
"disabled": true
},
{
"name": "merge_fix_fix_transpose",
"plugin": "vaip-pass_merge_fix_fix_transpose",
"enableGc": true,
"disabled": true
},
{
"name": "final_gc",
"plugin": "vaip-pass_remove_isolated_node"
}
],
"xcompilerAttrs": {
"debug_mode": {
"stringValue": "performance"
},
"dpu_subgraph_num": {
"uintValue": 32
},
"opt_level": {
"uintValue": 0
},
"dump_subgraph_ops": {
"boolValue": false
},
"profile": {
"uintValue": 0
},
"disable_std_quant": {
"boolValue": false
},
"enable_fast_pm": {
"boolValue": true
}
},
"minimum_num_of_conv": 2
}
}
],
"target": "RyzenAI_vision_config_1",
"targets": [
{
"name": "RyzenAI_vision_config_1",
"pass": [
"init",
"fuse_DPU"
]
}
]
}
15 changes: 0 additions & 15 deletions demo/multi-model-exec/env.yaml

This file was deleted.

Loading

0 comments on commit c301451

Please sign in to comment.