Update rknn-toolkit2/rknn-toolkit-lite2 to 1.5.2

Signed-off-by: Randall Zhuo <[email protected]>
fkiller · Aug 28, 2023 · b7c3b5c · b7c3b5c
1 parent d333c7d
commit b7c3b5c
Show file tree

Hide file tree

Showing 101 changed files with 1,109 additions and 121 deletions.
diff --git a/README.md b/README.md
@@ -23,7 +23,7 @@ Note:
 # Notes
 - Currently rknn-toolkit2 is not compatible with [rknn-toolkit](https://github.com/rockchip-linux/rknn-toolkit)
 - Currently only support on Ubuntu 18.04 python 3.6 / Ubuntu 20.04 python 3.8 / Ubuntu 22.04 python 3.10
-- Latest version:1.5.0(Release version)
+- Latest version:1.5.2(Release version)
 
 # Feedback and Community Support
 - [Redmine](https://redmine.rock-chips.com) (**Feedback recommended, Please consult our sales or FAE for the redmine account**)
@@ -32,4 +32,4 @@ Note:
 <center class="half">
   <img width="200" height="200"  src="https://github.com/rockchip-linux/rknn-toolkit/blob/master/QQGroupQRCode.png" title="QQ Group Chat"/>
   <img width="200" height="200"  src="https://github.com/rockchip-linux/rknn-toolkit/blob/master/QQGroup2QRCode.png" title="QQ Group Chat2"/>
-</center>
+</center>
diff --git a/Rockchip_Quick_Start_RKNN_SDK_V1.5.0_CN.pdf b/Rockchip_Quick_Start_RKNN_SDK_V1.5.0_CN.pdf
diff --git a/...it2_API_Difference_With_Toolkit1-1.5.0.md → ...it2_API_Difference_With_Toolkit1-1.5.2.md b/...it2_API_Difference_With_Toolkit1-1.5.0.md → ...it2_API_Difference_With_Toolkit1-1.5.2.md
diff --git a/doc/RKNNToolKit2_OP_Support-1.5.0.md → doc/RKNNToolKit2_OP_Support-1.5.2.md b/doc/RKNNToolKit2_OP_Support-1.5.0.md → doc/RKNNToolKit2_OP_Support-1.5.2.md
diff --git a/doc/RKNN_Compiler_Support_Operator_List_v1.5.0.pdf b/doc/RKNN_Compiler_Support_Operator_List_v1.5.0.pdf
diff --git a/doc/RKNN_Compiler_Support_Operator_List_v1.5.2.pdf b/doc/RKNN_Compiler_Support_Operator_List_v1.5.2.pdf
diff --git a/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.2_CN.pdf b/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.2_CN.pdf
diff --git a/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.2_EN.pdf b/doc/Rockchip_Quick_Start_RKNN_SDK_V1.5.2_EN.pdf
diff --git a/doc/Rockchip_Quick_Start_RKNN_Toolkit2_CN-1.5.0.pdf b/doc/Rockchip_Quick_Start_RKNN_Toolkit2_CN-1.5.0.pdf
diff --git a/doc/Rockchip_Quick_Start_RKNN_Toolkit2_CN-1.5.2.pdf b/doc/Rockchip_Quick_Start_RKNN_Toolkit2_CN-1.5.2.pdf
diff --git a/doc/Rockchip_Quick_Start_RKNN_Toolkit2_EN-1.5.0.pdf b/doc/Rockchip_Quick_Start_RKNN_Toolkit2_EN-1.5.0.pdf
diff --git a/doc/Rockchip_Quick_Start_RKNN_Toolkit2_EN-1.5.2.pdf b/doc/Rockchip_Quick_Start_RKNN_Toolkit2_EN-1.5.2.pdf
diff --git a/doc/Rockchip_Trouble_Shooting_RKNN_Toolkit2_CN-1.5.0.pdf b/doc/Rockchip_Trouble_Shooting_RKNN_Toolkit2_CN-1.5.0.pdf
diff --git a/doc/Rockchip_Trouble_Shooting_RKNN_Toolkit2_CN-1.5.2.pdf b/doc/Rockchip_Trouble_Shooting_RKNN_Toolkit2_CN-1.5.2.pdf
diff --git a/doc/Rockchip_Trouble_Shooting_RKNN_Toolkit2_EN-1.5.0.pdf b/doc/Rockchip_Trouble_Shooting_RKNN_Toolkit2_EN-1.5.0.pdf
diff --git a/doc/Rockchip_Trouble_Shooting_RKNN_Toolkit2_EN-1.5.2.pdf b/doc/Rockchip_Trouble_Shooting_RKNN_Toolkit2_EN-1.5.2.pdf
diff --git a/doc/Rockchip_User_Guide_RKNN_Toolkit2_CN-1.5.0.pdf b/doc/Rockchip_User_Guide_RKNN_Toolkit2_CN-1.5.0.pdf
diff --git a/doc/Rockchip_User_Guide_RKNN_Toolkit2_CN-1.5.2.pdf b/doc/Rockchip_User_Guide_RKNN_Toolkit2_CN-1.5.2.pdf
diff --git a/doc/Rockchip_User_Guide_RKNN_Toolkit2_EN-1.5.0.pdf b/doc/Rockchip_User_Guide_RKNN_Toolkit2_EN-1.5.0.pdf
diff --git a/doc/Rockchip_User_Guide_RKNN_Toolkit2_EN-1.5.2.pdf b/doc/Rockchip_User_Guide_RKNN_Toolkit2_EN-1.5.2.pdf
diff --git a/doc/changelog-1.5.0.txt → doc/changelog-1.5.2.txt b/doc/changelog-1.5.0.txt → doc/changelog-1.5.2.txt
@@ -1,3 +1,93 @@
+2023-8-25
+版本: v1.5.2:
+更新内容:
+1. 添加glu支持
+2. 更新examples
+3. 完善dynamic_input功能
+4. 优化transformer模型支持
+5. 添加torch的部分op支持
+6. 更新rknn_batch_size支持
+
+2023-7-14
+版本: v1.5.1b19:
+更新内容:
+1. 优化quantize_weight功能,优化部分网络生成的rknn模型大小.
+
+2023-7-3
+版本: v1.5.1b18:
+更新内容:
+1. 增加Matmul层GPU target支持
+2. 增加where+softmax子图优化
+3. 修复load_rknn在dynamic_input启用时的部分问题
+4. 添加新的图优化规则
+5. 更新对大模型动态输入的支持
+
+2023-6-29
+版本: v1.5.1b17:
+更新内容:
+1. 修复LSTM的重复调用rknn_inputs_set导致卡死问题
+2. 修复Transpose输出内存分配过大问题
+3. 优化部分单通道输入大分辨率模型效率
+4. 修复图优化规则报错问题
+5. 修复原生动态模型的大部分问题
+
+2023-6-25
+版本: v1.5.1b16:
+更新内容:
+1. 修复动态shape LSTM的错误问题
+2. 更新config.target_platform的默认值
+3. 修复原生动态模型的支持问题
+
+2023-6-16
+版本: v1.5.1b13:
+更新内容:
+1. 动态shape rknn_batch_size > 1的支持
+2. 更新动态输入的支持
+3. 更新部分图优化支持
+4. 添加对原生动态模型的初步支持
+
+2023-6-9
+版本: v1.5.1b10:
+更新内容:
+1. reshape/transpose等胶水算子的layout优化
+
+2023-6-9
+版本: v1.5.1b9:
+更新内容:
+1. 修复动态shape最高维度非1情况结果错误的bug
+
+2023-6-9
+版本: v1.5.1b8:
+更新内容:
+1. [C API]修复多核多batch模式运行结果错误的Bug
+2. [C API]优化rknn_init耗时
+3. [C API]增加CURRENT_NATIVE_INPUT/OUTPUT_ATTR属性查询
+4. 修复部分模型internal 内存偏大的问题
+5. 修复动态模型支持问题
+6. 动态模型添加对rknn_batch_size的支持
+7. 更新部分图优化支持
+
+2023-6-1
+版本: v1.5.1b3:
+更新内容:
+1. 修复动态shape性能分析/内存分析失败的Bug
+2. 降低导出RKNN模型大小
+3. 添加通过加载多个onnx模型来仿真动态模型的实验性功能
+
+2023-5-30
+版本: v1.5.1b2:
+更新内容:
+1. [新特性]采用预编译模型，提高Runtime初始化RKNN模型效率。要求重新导出RKNN模型并更新Runtime，若不使用新特性无需重新导出
+2. 修复稀疏化的报错和部分精度问题
+3. 更新cp38/cp310的依赖库版本
+4. 混合量化添加对子图进行混合量化的功能
+
+2023-5-27
+版本: v1.5.1b1:
+更新内容:
+1. 修复Pad算子的错误
+2. 添加Mul的广播部分支持
+
 2023-5-18
 版本: v1.5.0:
 更新内容:
@@ -7,7 +97,7 @@
 2023-5-17
 版本: v1.4.6b2:
 更新内容:
-1. 修复RK3562 多batch rknn模型C API运行错误的Bug。
+1. 修复RK3562 多batch rknn模型C API运行错误的Bug
 
 2023-5-15
 版本: v1.4.6b1:
@@ -77,7 +167,7 @@
 2023-4-14
 版本: v1.4.3b12:
 更新内容:
-1. [RK3562]增加指定层跑CPU/GPU/NPU特性。
+1. [RK3562]增加指定层跑CPU/GPU/NPU特性
 2. 修复concat优化规则
 3. 添加op_target功能
 
@@ -107,14 +197,14 @@
 2023-3-27
 版本: v1.4.3b3:
 更新内容:
-1. [RK3566]优化CNN+LSTM结构模型的内存。
+1. [RK3566]优化CNN+LSTM结构模型的内存
 2. 优化Concat性能
 3. load_tflite/load_tensorflow添加input_is_nchw参数
 
 2023-3-23
 版本: v1.4.3b2:
 更新内容:
-1. mul/add/div/sub算子优化。
+1. mul/add/div/sub算子优化
 2. 修复多级maxpool量化问题
 
 2023-3-21
@@ -140,7 +230,7 @@
 2023-3-9
 版本: v1.4.2b6:
 更新内容:
-1. RK3562平台Bug修复。
+1. RK3562平台Bug修复
 2. 增加model_pruning控制，并支持deconv，以及Bug修复
 3. 增加If/Loop的部分转换支持
 4. 修复MMSE部分模型失败的问题
@@ -164,26 +254,26 @@
 2023-2-8
 版本: v1.4.1b23:
 更新内容:
-1. 修复特定stride反卷积算子的Bug。
+1. 修复特定stride反卷积算子的Bug
 2. 更新MatMul的perchannel量化支持
 3. 更新动态图检测功能
 4. 优化where的量化支持
 
 2023-2-2
 版本: v1.4.1b22:
 更新内容:
-1. 增加Equal算子对Bool类型支持。
-2. 修复Matmul算子/exLayerNorm算子的Bug.
+1. 增加Equal算子对Bool类型支持
+2. 修复Matmul算子/exLayerNorm算子的Bug
 3. 更新equal/slice/cast/pad/ConvTranspose支持
 4. 更新QAT模型支持
 5. 移除bfloat16包依赖
 
 2023-1-13
 版本: v1.4.1b21:
 更新内容:
-1. 修复RK3588 Matmul接口错误。
-2. 修复4通道输入float16类型模型在RK356X平台查询虚宽错误问题。
-3. 模型不填写量化信息情况下，默认Tensor量化类型为float16。
+1. 修复RK3588 Matmul接口错误
+2. 修复4通道输入float16类型模型在RK356X平台查询虚宽错误问题
+3. 模型不填写量化信息情况下，默认Tensor量化类型为float16
 4. 增加unk__xxx无效shape支持
 5. 更新abs/dataconvert支持
 6. 优化模型剪枝功能

diff --git a/doc/requirements_cp310-1.5.0.txt → doc/requirements_cp310-1.5.2.txt b/doc/requirements_cp310-1.5.0.txt → doc/requirements_cp310-1.5.2.txt
@@ -15,7 +15,7 @@ opencv-python==4.5.5.64
 fast-histogram==0.11
 
 # base
-onnx==1.13.1
+onnx==1.14.0
 onnxoptimizer==0.3.8
 onnxruntime==1.14.1
 torch==1.13.1

diff --git a/doc/requirements_cp36-1.5.0.txt → doc/requirements_cp36-1.5.2.txt b/doc/requirements_cp36-1.5.0.txt → doc/requirements_cp36-1.5.2.txt
diff --git a/doc/requirements_cp38-1.5.0.txt → doc/requirements_cp38-1.5.2.txt b/doc/requirements_cp38-1.5.0.txt → doc/requirements_cp38-1.5.2.txt
@@ -1,8 +1,8 @@
 # if install failed, please change the pip source to 'https://mirror.baidu.com/pypi/simple'
 
 # base deps
-numpy==1.19.5
-protobuf==3.12.2
+numpy==1.21.6
+protobuf==3.20.3
 flatbuffers==1.12
 
 # utils
@@ -15,9 +15,9 @@ opencv-python==4.5.5.64
 fast-histogram==0.11
 
 # base
-onnx==1.10.0
+onnx==1.14.0
 onnxoptimizer==0.2.7
-onnxruntime==1.10.0
+onnxruntime==1.14.0
 torch==1.10.1
 torchvision==0.11.2
-tensorflow==2.6.2
+tensorflow==2.8.0
diff --git a/docker/docker_file/ubuntu_18_04_cp36/Dockerfile_ubuntu_18_04_for_cp36 b/docker/docker_file/ubuntu_18_04_cp36/Dockerfile_ubuntu_18_04_for_cp36
@@ -22,5 +22,5 @@ RUN pip3 config set install.trusted-host mirror.baidu.com
 
 RUN python3 --version
 RUN pip3 --version
-COPY rknn_toolkit2-1.5.0+1fa95b5c-cp36-cp36m-linux_x86_64.whl rknn_toolkit2-1.5.0+1fa95b5c-cp36-cp36m-linux_x86_64.whl
-RUN pip3 install rknn_toolkit2-1.5.0+1fa95b5c-cp36-cp36m-linux_x86_64.whl
+COPY rknn_toolkit2-1.5.2+b642f30c-cp36-cp36m-linux_x86_64.whl rknn_toolkit2-1.5.2+b642f30c-cp36-cp36m-linux_x86_64.whl
+RUN pip3 install rknn_toolkit2-1.5.2+b642f30c-cp36-cp36m-linux_x86_64.whl
diff --git a/...1.5.0+1fa95b5c-cp38-cp38-linux_x86_64.whl → ....5.2+b642f30c-cp36-cp36m-linux_x86_64.whl b/...1.5.0+1fa95b5c-cp38-cp38-linux_x86_64.whl → ....5.2+b642f30c-cp36-cp36m-linux_x86_64.whl
diff --git a/examples/caffe/mobilenet_v2/README.md b/examples/caffe/mobilenet_v2/README.md
@@ -0,0 +1,26 @@
+# Caffe MobileNet V2
+
+## Model Source
+The model used in this example come from the following open source projects:  
+https://github.com/shicai/MobileNet-Caffe
+
+## Script Usage
+*Usage:*
+```
+python test.py
+```
+*Description:*
+- The default target platform in script is 'rk3566', please modify the 'target_platform' parameter of 'rknn.config' according to the actual platform.
+- If connecting board is required, please add the 'target' parameter in 'rknn.init_runtime'.
+
+## Expected Results
+This example will print the TOP5 labels and corresponding scores of the test image classification results, as follows:
+```
+-----TOP 5-----
+[155]: 0.9931640625
+[154]: 0.00266265869140625
+[204]: 0.0019779205322265625
+[283]: 0.0009202957153320312
+[194]: 0.0001285076141357422
+```
+- Note: Different platforms, different versions of tools and drivers may have slightly different results.
diff --git a/.../caffe/mobilenet_v2/mobilenet_v2.prototxt → ...mobilenet_v2/mobilenet_v2_deploy.prototxt b/.../caffe/mobilenet_v2/mobilenet_v2.prototxt → ...mobilenet_v2/mobilenet_v2_deploy.prototxt
@@ -5,19 +5,11 @@ name: "MOBILENET_V2"
 #    crop_size: 224
 #    mean_value: [103.94,116.78,123.68]
 #  }
-layer {
-  name: "data"
-  type: "Input"
-  top: "data"
-  input_param {
-    shape {
-      dim: 1
-      dim: 3
-      dim: 224
-      dim: 224
-    }
-  }
-}
+input: "data"
+input_dim: 1
+input_dim: 3
+input_dim: 224
+input_dim: 224
 layer {
   name: "conv1"
   type: "Convolution"
@@ -3422,4 +3414,4 @@ layer {
   type: "Softmax"
   bottom: "fc7"
   top: "prob"
-}
+}
diff --git a/examples/caffe/mobilenet_v2/test.py b/examples/caffe/mobilenet_v2/test.py
@@ -29,12 +29,12 @@ def show_outputs(outputs):
 
     # Pre-process config
     print('--> Config model')
-    rknn.config(mean_values=[103.94, 116.78, 123.68], std_values=[58.82, 58.82, 58.82], quant_img_RGB2BGR=True)
+    rknn.config(mean_values=[103.94, 116.78, 123.68], std_values=[58.82, 58.82, 58.82], quant_img_RGB2BGR=True, target_platform='rk3566')
     print('done')
 
-    # Load model
+    # Load model (from https://github.com/shicai/MobileNet-Caffe)
     print('--> Loading model')
-    ret = rknn.load_caffe(model='./mobilenet_v2.prototxt',
+    ret = rknn.load_caffe(model='./mobilenet_v2_deploy.prototxt',
                           blobs='./mobilenet_v2.caffemodel')
     if ret != 0:
         print('Load model failed!')

diff --git a/examples/caffe/vgg-ssd/README.md b/examples/caffe/vgg-ssd/README.md
@@ -0,0 +1,20 @@
+# Caffe VGG-SSD
+
+## Model Source
+The model used in this example come from the following open source projects:  
+https://github.com/weiliu89/caffe/tree/ssd#models  
+But the model download link has expired, please download from https://eyun.baidu.com/s/3jJhPRzo, password is 'rknn'.
+
+## Script Usage
+*Usage:*
+```
+python test.py
+```
+*Description:*
+- The default target platform in script is 'rk3566', please modify the 'target_platform' parameter of 'rknn.config' according to the actual platform.
+- If connecting board is required, please add the 'target' parameter in 'rknn.init_runtime'.
+
+## Expected Results
+This example will save the result of object detection to the 'result.jpg', as follows:  
+![result](result_truth.jpg)
+- Note: Different platforms, different versions of tools and drivers may have slightly different results.
diff --git a/examples/caffe/vgg-ssd/result_truth.jpg b/examples/caffe/vgg-ssd/result_truth.jpg
diff --git a/examples/caffe/vgg-ssd/test.py b/examples/caffe/vgg-ssd/test.py
@@ -161,7 +161,7 @@ def ssd_post_process(conf_data, loc_data):
 
     # Pre-process config
     print('--> Config model')
-    rknn.config(mean_values=[103.94, 116.78, 123.68], std_values=[1, 1, 1], quant_img_RGB2BGR=True)
+    rknn.config(mean_values=[103.94, 116.78, 123.68], std_values=[1, 1, 1], quant_img_RGB2BGR=True, target_platform='rk3566')
     print('done')
 
     # Load model

diff --git a/examples/darknet/yolov3_416x416/README.md b/examples/darknet/yolov3_416x416/README.md
@@ -0,0 +1,20 @@
+# Darknet YOLO V3
+
+## Model Source
+The model used in this example come from the following open source projects:  
+https://pjreddie.com/darknet/yolo/  
+The 'yolov3.cfg' changed the width & height from 608 to 416 on the basis of https://github.com/pjreddie/darknet/blob/master/cfg/yolov3.cfg
+
+## Script Usage
+*Usage:*
+```
+python test.py
+```
+*Description:*
+- The default target platform in script is 'rk3566', please modify the 'target_platform' parameter of 'rknn.config' according to the actual platform.
+- If connecting board is required, please add the 'target' parameter in 'rknn.init_runtime'.
+
+## Expected Results
+This example will save the result of object detection to the 'result.jpg', as follows:  
+![result](result_truth.jpg)
+- Note: Different platforms, different versions of tools and drivers may have slightly different results.
diff --git a/examples/darknet/yolov3_416x416/result_truth.jpg b/examples/darknet/yolov3_416x416/result_truth.jpg
diff --git a/examples/darknet/yolov3_416x416/test.py b/examples/darknet/yolov3_416x416/test.py
@@ -12,6 +12,9 @@
 
 if __name__ == '__main__':
 
+    # Model from https://pjreddie.com/darknet/yolo/
+    # The yolov3.cfg changed the width & height from 608 to 416 on the basis of 
+    # https://github.com/pjreddie/darknet/blob/master/cfg/yolov3.cfg
     MODEL_PATH = './yolov3.cfg'
     WEIGHT_PATH = './yolov3.weights'
     RKNN_MODEL_PATH = './yolov3_416.rknn'
@@ -26,7 +29,7 @@
 
     # Pre-process config
     print('--> Config model')
-    rknn.config(mean_values=[0, 0, 0], std_values=[255, 255, 255])
+    rknn.config(mean_values=[0, 0, 0], std_values=[255, 255, 255], target_platform='rk3566')
     print('done')
 
     # Load model
@@ -92,7 +95,7 @@
     if boxes is not None:
         draw(image, boxes, scores, classes)
 
-    print('Save results to results.jpg!')
+    print('Save results to result.jpg!')
     cv2.imwrite('result.jpg', image)
 
     rknn.release()