[Other] Add detection, segmentation and OCR examples for Ascend deplo…

…y. (PaddlePaddle#983) * Add Huawei Ascend NPU deploy through PaddleLite CANN * Add NNAdapter interface for paddlelite * Modify Huawei Ascend Cmake * Update way for compiling Huawei Ascend NPU deployment * remove UseLiteBackend in UseCANN * Support compile python whlee * Change names of nnadapter API * Add nnadapter pybind and remove useless API * Support Python deployment on Huawei Ascend NPU * Add models suppor for ascend * Add PPOCR rec reszie for ascend * fix conflict for ascend * Rename CANN to Ascend * Rename CANN to Ascend * Improve ascend * fix ascend bug * improve ascend docs * improve ascend docs * improve ascend docs * Improve Ascend * Improve Ascend * Move ascend python demo * Imporve ascend * Improve ascend * Improve ascend * Improve ascend * Improve ascend * Imporve ascend * Imporve ascend * Improve ascend * acc eval script * acc eval * remove acc_eval from branch huawei * Add detection and segmentation examples for Ascend deployment * Add detection and segmentation examples for Ascend deployment * Add PPOCR example for ascend deploy * Imporve paddle lite compiliation * Add FlyCV doc * Add FlyCV doc * Add FlyCV doc * Imporve Ascend docs * Imporve Ascend docs
heliqi · Jan 4, 2023 · 58d63f3 · 58d63f3
1 parent 2cfd331
commit 58d63f3
Show file tree

Hide file tree

Showing 57 changed files with 694 additions and 93 deletions.
diff --git a/cmake/ascend.cmake b/cmake/ascend.cmake
@@ -16,13 +16,4 @@ else ()
   if(NOT PADDLELITE_URL)
     set(PADDLELITE_URL "https://bj.bcebos.com/fastdeploy/test/lite-linux_arm64_huawei_ascend_npu_python_1207.tgz")
   endif()
-  execute_process(COMMAND sh -c "ls *.so*" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/third_libs/install/paddlelite/lib
-  COMMAND sh -c "xargs ${PATCHELF_EXE} --set-rpath '$ORIGIN'" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/third_libs/install/paddlelite/lib
-          RESULT_VARIABLE result
-                OUTPUT_VARIABLE curr_out
-                ERROR_VARIABLE  curr_out)
-  if(ret EQUAL "1")
-    message(FATAL_ERROR "Failed to patchelf Paddle Lite libraries when using Ascend.")
-  endif()
-  message(STATUS "result:${result} out:${curr_out}")
 endif()	
diff --git a/cmake/paddlelite.cmake b/cmake/paddlelite.cmake
@@ -114,6 +114,10 @@ else()
     BUILD_BYPRODUCTS ${PADDLELITE_LIB})
 endif()  
 
+if(UNIX AND (NOT APPLE) AND (NOT ANDROID))
+  add_custom_target(patchelf_paddle_lite ALL COMMAND  bash -c "PATCHELF_EXE=${PATCHELF_EXE} python ${PROJECT_SOURCE_DIR}/scripts/patch_paddle_lite.py ${PADDLELITE_INSTALL_DIR}/lib/" DEPENDS ${LIBRARY_NAME})
+endif()
+
 add_library(external_paddle_lite STATIC IMPORTED GLOBAL)
 set_property(TARGET external_paddle_lite PROPERTY IMPORTED_LOCATION ${PADDLELITE_LIB})
 add_dependencies(external_paddle_lite ${PADDLELITE_PROJECT})
diff --git a/docs/cn/build_and_install/huawei_ascend.md b/docs/cn/build_and_install/huawei_ascend.md
@@ -94,6 +94,12 @@ python setup.py bdist_wheel
 
 #编译完成后,请用户自行安装当前目录的dist文件夹内的whl包.
 ```
+## 五.昇腾部署时开启FlyCV
+[FlyCV](https://github.com/PaddlePaddle/FlyCV) 是一款高性能计算机图像处理库, 针对ARM架构做了很多优化, 相比其他图像处理库性能更为出色.
+FastDeploy现在已经集成FlyCV, 用户可以在支持的硬件平台上使用FlyCV, 实现模型端到端推理性能的加速.
+模型端到端推理中, 预处理和后处理阶段为CPU计算, 当用户使用ARM CPU + 昇腾的硬件平台时, 我们推荐用户使用FlyCV, 可以实现端到端的推理性能加速, 详见[FLyCV使用文档](./boost_cv_by_flycv.md).
 
-- 华为昇腾NPU 上使用C++部署 PaddleClas 分类模型请参考：[PaddleClas 华为升腾NPU C++ 部署示例](../../../examples/vision/classification/paddleclas/ascend/cpp/README.md)
-- 华为昇腾NPU 上使用Python部署 PaddleClas 分类模型请参考：[PaddleClas 华为升腾NPU Python 部署示例](../../../examples/vision/classification/paddleclas/ascend/python/README.md)
+
+## 六.昇腾部署Demo参考
+- 华为昇腾NPU 上使用C++部署 PaddleClas 分类模型请参考：[PaddleClas 华为升腾NPU C++ 部署示例](../../../examples/vision/classification/paddleclas/cpp/README.md)
+- 华为昇腾NPU 上使用Python部署 PaddleClas 分类模型请参考：[PaddleClas 华为升腾NPU Python 部署示例](../../../examples/vision/classification/paddleclas/python/README.md)
diff --git a/docs/cn/faq/boost_cv_by_flycv.md b/docs/cn/faq/boost_cv_by_flycv.md
@@ -0,0 +1,68 @@
+[English](../../en/faq/boost_cv_by_flycv.md) | 中文
+
+
+# 使用FlyCV加速端到端推理性能
+
+[FlyCV](https://github.com/PaddlePaddle/FlyCV) 是一款高性能计算机图像处理库, 针对ARM架构做了很多优化, 相比其他图像处理库性能更为出色.
+FastDeploy现在已经集成FlyCV, 用户可以在支持的硬件平台上使用FlyCV, 实现模型端到端推理性能的加速.
+
+## 已支持的系统与硬件架构
+
+| 系统 | 硬件架构 |
+| :-----------| :--------   |
+|   Android     |  armeabi-v7a, arm64-v8a |  
+|   Linux       |  aarch64, armhf, x86_64|  
+
+
+## 使用方式
+使用FlyCV,首先需要在编译时开启FlyCV编译选项,之后在部署时新增一行代码即可开启.
+本文以Linux系统为例,说明如何开启FlyCV编译选项, 之后在部署时, 新增一行代码使用FlyCV.
+
+用户可以按照如下方式,在编译预测库时,开启FlyCV编译选项.
+```bash
+# 编译C++预测库时, 开启FlyCV编译选项.
+-DENABLE_VISION=ON \
+
+# 在编译Python预测库时, 开启FlyCV编译选项
+export ENABLE_FLYCV=ON
+```
+
+用户可以按照如下方式,在部署代码中新增一行代码启用FlyCV.
+```bash
+# C++部署代码.
+# 新增一行代码启用FlyCV
+fastdeploy::vision::EnableFlyCV();
+# 其他部署代码...(以昇腾部署为例)
+fastdeploy::RuntimeOption option;
+option.UseAscend();
+...
+
+
+# Python部署代码
+# 新增一行代码启用FlyCV
+fastdeploy.vision.enable_flycv()
+# 其他部署代码...(以昇腾部署为例)
+runtime_option = build_option()
+option.use_ascend()
+...
+```
+
+## 部分平台FlyCV 端到端性能数据
+
+鲲鹏920 CPU + Atlas 300I Pro 推理卡.
+| 模型 | OpenCV 端到端性能(ms) | FlyCV 端到端性能(ms) |  
+| :-----------| :--------   | :--------   |
+|   ResNet50     | 2.78  | 1.63  |  
+|   PP-LCNetV2   |  2.50 |  1.39   |  
+|   YOLOv7       |  27.00 | 21.36    |  
+|   PP_HumanSegV2_Lite   | 2.76 |  2.10   |  
+
+
+瑞芯微RV1126.
+
+| 模型 | OpenCV 端到端性能(ms) | FlyCV 端到端性能(ms) |  
+| :-----------| :--------   | :--------   |
+|   ResNet50     | 9.23  | 6.01  |  
+|   mobilenetv1_ssld_量化模型   |  9.23 |  6.01   |  
+|   yolov5s_量化模型       |  28.33 | 14.25    |  
+|   PP_LiteSeg_量化模型  | 132.25 |  60.31   |  
diff --git a/docs/cn/faq/use_sdk_on_ascend.md b/docs/cn/faq/use_sdk_on_ascend.md
@@ -1,4 +1,4 @@
-[English](../../en/faq/use_sdk_on_linux.md) | 中文
+[English](../../en/faq/use_sdk_on_ascend.md) | 中文
 
 
 # Linux上使用C++在华为昇腾部署

diff --git a/docs/en/build_and_install/huawei_ascend.md b/docs/en/build_and_install/huawei_ascend.md
@@ -93,6 +93,14 @@ python setup.py bdist_wheel
 #After the compilation is complete, please install the whl package in the dist folder of the current directory.
 ```
 
-Deploying PaddleClas Classification Model on Huawei Ascend NPU using C++ please refer to: [PaddleClas Huawei Ascend NPU C++ Deployment Example](../../../examples/vision/classification/paddleclas/ascend/cpp/README.md)
+## Enable FlyCV for Ascend deployment
 
-Deploying PaddleClas classification model on Huawei Ascend NPU using Python please refer to: [PaddleClas Huawei Ascend NPU Python Deployment Example](../../../examples/vision/classification/paddleclas/ascend/python/README.md)
+[FlyCV](https://github.com/PaddlePaddle/FlyCV) is a high performance computer image processing library, providing better performance than other image processing libraries, especially in the ARM architecture.
+FastDeploy is now integrated with FlyCV, allowing users to use FlyCV on supported hardware platforms to accelerate model end-to-end inference performance.
+In end-to-end model inference, the pre-processing and post-processing phases are CPU computation, we recommend using FlyCV for end-to-end inference performance acceleration when you are using ARM CPU + Ascend hardware platform. See [Enable FlyCV](./boost_cv_by_flycv.md) documentation for details.
+
+
+## Deployment demo reference
+- Deploying PaddleClas Classification Model on Huawei Ascend NPU using C++ please refer to: [PaddleClas Huawei Ascend NPU C++ Deployment Example](../../../examples/vision/classification/paddleclas/cpp/README.md)
+
+- Deploying PaddleClas classification model on Huawei Ascend NPU using Python please refer to: [PaddleClas Huawei Ascend NPU Python Deployment Example](../../../examples/vision/classification/paddleclas/python/README.md)
diff --git a/docs/en/faq/boost_cv_by_flycv.md b/docs/en/faq/boost_cv_by_flycv.md
@@ -0,0 +1,66 @@
+[简体中文](../../cn/faq/boost_cv_by_flycv.md) | English
+
+
+# Accelerate end-to-end inference performance using FlyCV
+
+[FlyCV](https://github.com/PaddlePaddle/FlyCV) is a high performance computer image processing library, providing better performance than other image processing libraries, especially in the ARM architecture.
+FastDeploy is now integrated with FlyCV, allowing users to use FlyCV on supported hardware platforms to accelerate model end-to-end inference performance.
+
+## Supported OS and Architectures
+
+| OS | Architectures |
+| :-----------| :--------   |
+|   Android     |  armeabi-v7a, arm64-v8a |  
+|   Linux       |  aarch64, armhf, x86_64|  
+
+
+## Usage
+To use FlyCV, you first need to turn on the FlyCV compile option at compile time, and then add a new line of code to turn it on.
+This article uses Linux as an example to show how to enable the FlyCV compile option, and then add a new line of code to use FlyCV during deployment.
+
+You can turn on the FlyCV compile option when compiling the FastDeploy library as follows.
+```bash
+# When compiling C++ libraries
+-DENABLE_VISION=ON
+
+#  When compiling Python libraries
+export ENABLE_FLYCV=ON
+```
+
+You can enable FlyCV by adding a new line of code to the deployment code as follows.
+```bash
+# C++ code
+fastdeploy::vision::EnableFlyCV();
+# Other..(e.g. With Huawei Ascend)
+fastdeploy::RuntimeOption option;
+option.UseAscend();
+...
+
+
+# Python code
+fastdeploy.vision.enable_flycv()
+# Other..(e.g. With Huawei Ascend)
+runtime_option = build_option()
+option.use_ascend()
+...
+```
+
+## Some Platforms FlyCV End-to-End Inference Performance
+
+KunPeng 920 CPU + Atlas 300I Pro.
+| Model | OpenCV E2E Performance(ms) | FlyCV E2E Performance(ms) |  
+| :-----------| :--------   | :--------   |
+|   ResNet50     | 2.78  | 1.63  |  
+|   PP-LCNetV2   |  2.50 |  1.39   |  
+|   YOLOv7       |  27.00 | 21.36    |  
+|   PP_HumanSegV2_Lite   | 2.76 |  2.10   |  
+
+
+Rockchip RV1126.
+
+| Model | OpenCV E2E Performance(ms) | FlyCV E2E Performance(ms) |  
+| :-----------| :--------   | :--------   |
+|   ResNet50     | 9.23  | 6.01  |  
+|   mobilenetv1_ssld_量化模型   |  9.23 |  6.01   |  
+|   yolov5s_量化模型       |  28.33 | 14.25    |  
+|   PP_LiteSeg_量化模型  | 132.25 |  60.31   |  
diff --git a/docs/en/faq/use_sdk_on_ascend.md b/docs/en/faq/use_sdk_on_ascend.md
@@ -1,4 +1,4 @@
-[简体中文](../../cn/faq/use_sdk_on_linux.md) | English
+[简体中文](../../cn/faq/use_sdk_on_ascend.md) | English
 
 
 # # Linux deployment with C++ on Huawei Ascend

diff --git a/examples/vision/detection/paddledetection/cpp/README.md b/examples/vision/detection/paddledetection/cpp/README.md
@@ -34,6 +34,8 @@ tar xvf ppyoloe_crn_l_300e_coco.tgz
 ./infer_ppyoloe_demo ./ppyoloe_crn_l_300e_coco 000000014439.jpg 2
 # 昆仑芯XPU推理
 ./infer_ppyoloe_demo ./ppyoloe_crn_l_300e_coco 000000014439.jpg 3
+# 华为昇腾推理
+./infer_ppyoloe_demo ./ppyoloe_crn_l_300e_coco 000000014439.jpg 4
 ```
 
 以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:  

diff --git a/examples/vision/detection/paddledetection/cpp/infer_ppyolo.cc b/examples/vision/detection/paddledetection/cpp/infer_ppyolo.cc
@@ -102,6 +102,33 @@ void GpuInfer(const std::string& model_dir, const std::string& image_file) {
   std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
 }
 
+void AscendInfer(const std::string& model_dir, const std::string& image_file) {
+  auto model_file = model_dir + sep + "model.pdmodel";
+  auto params_file = model_dir + sep + "model.pdiparams";
+  auto config_file = model_dir + sep + "infer_cfg.yml";
+  auto option = fastdeploy::RuntimeOption();
+  option.UseAscend();
+  auto model = fastdeploy::vision::detection::PPYOLO(model_file, params_file,
+                                                     config_file, option);
+  if (!model.Initialized()) {
+    std::cerr << "Failed to initialize." << std::endl;
+    return;
+  }
+
+  auto im = cv::imread(image_file);
+
+  fastdeploy::vision::DetectionResult res;
+  if (!model.Predict(im, &res)) {
+    std::cerr << "Failed to predict." << std::endl;
+    return;
+  }
+
+  std::cout << res.Str() << std::endl;
+  auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
+  cv::imwrite("vis_result.jpg", vis_im);
+  std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
+}
+
 int main(int argc, char* argv[]) {
   if (argc < 4) {
     std::cout
@@ -120,6 +147,8 @@ int main(int argc, char* argv[]) {
     GpuInfer(argv[1], argv[2]);
   } else if (std::atoi(argv[3]) == 2) {
     KunlunXinInfer(argv[1], argv[2]);
+  } else if (std::atoi(argv[3]) == 3) {
+    AscendInfer(argv[1], argv[2]);
   }
   return 0;
 }
diff --git a/examples/vision/detection/paddledetection/cpp/infer_ppyoloe.cc b/examples/vision/detection/paddledetection/cpp/infer_ppyoloe.cc
@@ -131,6 +131,33 @@ void TrtInfer(const std::string& model_dir, const std::string& image_file) {
   std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
 }
 
+void AscendInfer(const std::string& model_dir, const std::string& image_file) {
+  auto model_file = model_dir + sep + "model.pdmodel";
+  auto params_file = model_dir + sep + "model.pdiparams";
+  auto config_file = model_dir + sep + "infer_cfg.yml";
+  auto option = fastdeploy::RuntimeOption();
+  option.UseAscend();
+  auto model = fastdeploy::vision::detection::PPYOLOE(model_file, params_file,
+                                                      config_file, option);
+  if (!model.Initialized()) {
+    std::cerr << "Failed to initialize." << std::endl;
+    return;
+  }
+
+  auto im = cv::imread(image_file);
+
+  fastdeploy::vision::DetectionResult res;
+  if (!model.Predict(im, &res)) {
+    std::cerr << "Failed to predict." << std::endl;
+    return;
+  }
+
+  std::cout << res.Str() << std::endl;
+  auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
+  cv::imwrite("vis_result.jpg", vis_im);
+  std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
+}
+
 int main(int argc, char* argv[]) {
   if (argc < 4) {
     std::cout
@@ -151,6 +178,8 @@ int main(int argc, char* argv[]) {
     TrtInfer(argv[1], argv[2]);
   } else if (std::atoi(argv[3]) == 3) {
     KunlunXinInfer(argv[1], argv[2]);
+  } else if (std::atoi(argv[3]) == 4) {
+    AscendInfer(argv[1], argv[2]);
   }
   return 0;
 }
diff --git a/examples/vision/detection/paddledetection/cpp/infer_ssd.cc b/examples/vision/detection/paddledetection/cpp/infer_ssd.cc
@@ -104,6 +104,33 @@ void GpuInfer(const std::string& model_dir, const std::string& image_file) {
   std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
 }
 
+void AscendInfer(const std::string& model_dir, const std::string& image_file) {
+  auto model_file = model_dir + sep + "model.pdmodel";
+  auto params_file = model_dir + sep + "model.pdiparams";
+  auto config_file = model_dir + sep + "infer_cfg.yml";
+  auto option = fastdeploy::RuntimeOption();
+  option.UseAscend();
+  auto model = fastdeploy::vision::detection::SSD(model_file, params_file,
+                                                     config_file, option);
+  if (!model.Initialized()) {
+    std::cerr << "Failed to initialize." << std::endl;
+    return;
+  }
+
+  auto im = cv::imread(image_file);
+
+  fastdeploy::vision::DetectionResult res;
+  if (!model.Predict(im, &res)) {
+    std::cerr << "Failed to predict." << std::endl;
+    return;
+  }
+
+  std::cout << res.Str() << std::endl;
+  auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
+  cv::imwrite("vis_result.jpg", vis_im);
+  std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
+}
+
 int main(int argc, char* argv[]) {
   if (argc < 4) {
     std::cout
@@ -122,6 +149,8 @@ int main(int argc, char* argv[]) {
     GpuInfer(argv[1], argv[2]);
   } else if (std::atoi(argv[3]) == 2) {
     KunlunXinInfer(argv[1], argv[2]);
+  } else if (std::atoi(argv[3]) == 3) {
+    AscendInfer(argv[1], argv[2]);
   }
   return 0;
 }

diff --git a/examples/vision/detection/paddledetection/cpp/infer_yolov3.cc b/examples/vision/detection/paddledetection/cpp/infer_yolov3.cc
@@ -102,6 +102,34 @@ void GpuInfer(const std::string& model_dir, const std::string& image_file) {
   std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
 }
 
+
+void AscendInfer(const std::string& model_dir, const std::string& image_file) {
+  auto model_file = model_dir + sep + "model.pdmodel";
+  auto params_file = model_dir + sep + "model.pdiparams";
+  auto config_file = model_dir + sep + "infer_cfg.yml";
+  auto option = fastdeploy::RuntimeOption();
+  option.UseAscend();
+  auto model = fastdeploy::vision::detection::YOLOv3(model_file, params_file,
+                                                     config_file, option);
+  if (!model.Initialized()) {
+    std::cerr << "Failed to initialize." << std::endl;
+    return;
+  }
+
+  auto im = cv::imread(image_file);
+
+  fastdeploy::vision::DetectionResult res;
+  if (!model.Predict(im, &res)) {
+    std::cerr << "Failed to predict." << std::endl;
+    return;
+  }
+
+  std::cout << res.Str() << std::endl;
+  auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
+  cv::imwrite("vis_result.jpg", vis_im);
+  std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
+}
+
 int main(int argc, char* argv[]) {
   if (argc < 4) {
     std::cout
@@ -120,6 +148,8 @@ int main(int argc, char* argv[]) {
     GpuInfer(argv[1], argv[2]);
   } else if (std::atoi(argv[3]) == 2) {
     KunlunXinInfer(argv[1], argv[2]);
+  } else if (std::atoi(argv[3]) == 3) {
+    AscendInfer(argv[1], argv[2]);
   }
   return 0;
 }
diff --git a/examples/vision/detection/paddledetection/python/README.md b/examples/vision/detection/paddledetection/python/README.md
@@ -25,6 +25,8 @@ python infer_ppyoloe.py --model_dir ppyoloe_crn_l_300e_coco --image 000000014439
 python infer_ppyoloe.py --model_dir ppyoloe_crn_l_300e_coco --image 000000014439.jpg --device gpu --use_trt True
 # 昆仑芯XPU推理
 python infer_ppyoloe.py --model_dir ppyoloe_crn_l_300e_coco --image 000000014439.jpg --device kunlunxin
+# 华为昇腾推理
+python infer_ppyoloe.py --model_dir ppyoloe_crn_l_300e_coco --image 000000014439.jpg --device ascend
 ```
 
 运行完成可视化结果如下图所示