forked from PaddlePaddle/FastDeploy
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
4ccfbea
commit f900199
Showing
6 changed files
with
239 additions
and
1 deletion.
There are no files selected for viewing
55 changes: 55 additions & 0 deletions
55
examples/vision/keypointdetection/tiny_pose/rknpu2/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
[English](README.md) | 简体中文 | ||
# PP-TinyPose RKNPU2部署示例 | ||
|
||
## 模型版本说明 | ||
|
||
- [PaddleDetection release/2.5](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.5) | ||
|
||
目前FastDeploy支持如下模型的部署 | ||
|
||
- [PP-TinyPose系列模型](https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.5/configs/keypoint/tiny_pose/README.md) | ||
|
||
## 准备PP-TinyPose部署模型 | ||
|
||
PP-TinyPose模型导出,请参考其文档说明[模型导出](https://github.com/PaddlePaddle/PaddleDetection/blob/release/2.5/deploy/EXPORT_MODEL.md) | ||
|
||
**注意**:PP-TinyPose导出的模型包含`model.pdmodel`、`model.pdiparams`和`infer_cfg.yml`三个文件,FastDeploy会从yaml文件中获取模型在推理时需要的预处理信息。 | ||
|
||
## 模型转换example | ||
|
||
### Paddle模型转换为ONNX模型 | ||
|
||
由于Rockchip提供的rknn-toolkit2工具暂时不支持Paddle模型直接导出为RKNN模型,因此需要先将Paddle模型导出为ONNX模型,再将ONNX模型转为RKNN模型。 | ||
|
||
```bash | ||
# 下载Paddle静态图模型并解压 | ||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_TinyPose_256x192_infer.tgz | ||
tar -xvf PP_TinyPose_256x192_infer.tgz | ||
|
||
# 静态图转ONNX模型,注意,这里的save_file请和压缩包名对齐 | ||
paddle2onnx --model_dir PP_TinyPose_256x192_infer \ | ||
--model_filename model.pdmodel \ | ||
--params_filename model.pdiparams \ | ||
--save_file PP_TinyPose_256x192_infer/PP_TinyPose_256x192_infer.onnx \ | ||
--enable_dev_version True | ||
|
||
# 固定shape | ||
python -m paddle2onnx.optimize --input_model PP_TinyPose_256x192_infer/PP_TinyPose_256x192_infer.onnx \ | ||
--output_model PP_TinyPose_256x192_infer/PP_TinyPose_256x192_infer.onnx \ | ||
--input_shape_dict "{'image':[1,3,256,192]}" | ||
``` | ||
|
||
### ONNX模型转RKNN模型 | ||
|
||
为了方便大家使用,我们提供了python脚本,通过我们预配置的config文件,你将能够快速地转换ONNX模型到RKNN模型 | ||
|
||
```bash | ||
python tools/rknpu2/export.py --config_path tools/rknpu2/config/PP_TinyPose_256x192_unquantized.yaml \ | ||
--target_platform rk3588 | ||
``` | ||
|
||
## 详细部署文档 | ||
|
||
- [模型详细介绍](../README_CN.md) | ||
- [Python部署](python) | ||
- [C++部署](cpp) |
13 changes: 13 additions & 0 deletions
13
examples/vision/keypointdetection/tiny_pose/rknpu2/cpp/CMakeLists.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
PROJECT(infer_demo C CXX) | ||
CMAKE_MINIMUM_REQUIRED (VERSION 3.12) | ||
|
||
# 指定下载解压后的fastdeploy库路径 | ||
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.") | ||
|
||
include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake) | ||
|
||
# 添加FastDeploy依赖头文件 | ||
include_directories(${FASTDEPLOY_INCS}) | ||
|
||
add_executable(infer_tinypose_demo ${PROJECT_SOURCE_DIR}/pptinypose_infer.cc) | ||
target_link_libraries(infer_tinypose_demo ${FASTDEPLOY_LIBS}) |
85 changes: 85 additions & 0 deletions
85
examples/vision/keypointdetection/tiny_pose/rknpu2/cpp/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
[English](README.md) | 简体中文 | ||
# PP-TinyPose C++部署示例 | ||
|
||
本目录下提供`pptinypose_infer.cc`快速完成PP-TinyPose通过NPU加速部署的`单图单人关键点检测`示例 | ||
>> **注意**: PP-Tinypose单模型目前只支持单图单人关键点检测,因此输入的图片应只包含一个人或者进行过裁剪的图像。多人关键点检测请参考[PP-TinyPose Pipeline](../../../det_keypoint_unite/cpp/README.md) | ||
在部署前,需确认以下两个步骤 | ||
|
||
- 1. 软硬件环境满足要求,参考[FastDeploy环境要求](../../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md) | ||
- 2. 根据开发环境,下载预编译部署库和samples代码,参考[FastDeploy预编译库](../../../../../../docs/cn/build_and_install/download_prebuilt_libraries.md) | ||
|
||
|
||
以Linux上推理为例,在本目录执行如下命令即可完成编译测试,支持此模型需保证FastDeploy版本1.0.3以上(x.x.x>=1.0.3) | ||
|
||
```bash | ||
mkdir build | ||
cd build | ||
# 下载FastDeploy预编译库,用户可在上文提到的`FastDeploy预编译库`中自行选择合适的版本使用 | ||
wget https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-x64-x.x.x.tgz | ||
tar xvf fastdeploy-linux-x64-x.x.x.tgz | ||
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-x.x.x | ||
make -j | ||
|
||
# 下载PP-TinyPose模型文件和测试图片 | ||
wget https://bj.bcebos.com/paddlehub/fastdeploy/PP_TinyPose_256x192_infer.tgz | ||
tar -xvf PP_TinyPose_256x192_infer.tgz | ||
wget https://bj.bcebos.com/paddlehub/fastdeploy/hrnet_demo.jpg | ||
|
||
|
||
# CPU推理 | ||
./infer_tinypose_demo PP_TinyPose_256x192_infer hrnet_demo.jpg | ||
``` | ||
|
||
运行完成可视化结果如下图所示 | ||
<div align="center"> | ||
<img src="https://user-images.githubusercontent.com/16222477/196386764-dd51ad56-c410-4c54-9580-643f282f5a83.jpeg", width=359px, height=423px /> | ||
</div> | ||
|
||
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考: | ||
- [如何在Windows中使用FastDeploy C++ SDK](../../../../../docs/cn/faq/use_sdk_on_windows.md) | ||
|
||
## PP-TinyPose C++接口 | ||
|
||
### PP-TinyPose类 | ||
|
||
```c++ | ||
fastdeploy::vision::keypointdetection::PPTinyPose( | ||
const string& model_file, | ||
const string& params_file = "", | ||
const string& config_file, | ||
const RuntimeOption& runtime_option = RuntimeOption(), | ||
const ModelFormat& model_format = ModelFormat::PADDLE) | ||
``` | ||
PPTinyPose模型加载和初始化,其中model_file为导出的Paddle模型格式。 | ||
**参数** | ||
> * **model_file**(str): 模型文件路径 | ||
> * **params_file**(str): 参数文件路径 | ||
> * **config_file**(str): 推理部署配置文件 | ||
> * **runtime_option**(RuntimeOption): 后端推理配置,默认为None,即采用默认配置 | ||
> * **model_format**(ModelFormat): 模型格式,默认为Paddle格式 | ||
#### Predict函数 | ||
> ```c++ | ||
> PPTinyPose::Predict(cv::Mat* im, KeyPointDetectionResult* result) | ||
> ``` | ||
> | ||
> 模型预测接口,输入图像直接输出关键点检测结果。 | ||
> | ||
> **参数** | ||
> | ||
> > * **im**: 输入图像,注意需为HWC,BGR格式 | ||
> > * **result**: 关键点检测结果,包括关键点的坐标以及关键点对应的概率值, KeyPointDetectionResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/) | ||
### 类成员属性 | ||
#### 后处理参数 | ||
> > * **use_dark**(bool): 是否使用DARK进行后处理[参考论文](https://arxiv.org/abs/1910.06278) | ||
- [模型介绍](../../) | ||
- [Python部署](../python) | ||
- [视觉模型预测结果](../../../../../docs/api/vision_results/) | ||
- [如何切换模型推理后端引擎](../../../../../docs/cn/faq/how_to_change_backend.md) |
70 changes: 70 additions & 0 deletions
70
examples/vision/keypointdetection/tiny_pose/rknpu2/cpp/pptinypose_infer.cc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
#include "fastdeploy/vision.h" | ||
|
||
void RKNPU2Infer(const std::string& tinypose_model_dir, | ||
const std::string& image_file) { | ||
auto tinypose_model_file = | ||
tinypose_model_dir + "/picodet_s_416_coco_lcnet_rk3588.rknn"; | ||
auto tinypose_params_file = ""; | ||
auto tinypose_config_file = tinypose_model_dir + "infer_cfg.yml"; | ||
auto option = fastdeploy::RuntimeOption(); | ||
option.UseRKNPU2(); | ||
auto tinypose_model = fastdeploy::vision::keypointdetection::PPTinyPose( | ||
tinypose_model_file, tinypose_params_file, tinypose_config_file, option); | ||
|
||
if (!tinypose_model.Initialized()) { | ||
std::cerr << "TinyPose Model Failed to initialize." << std::endl; | ||
return; | ||
} | ||
|
||
tinypose_model.DisablePermute(); | ||
tinypose_model.DisableNormalize(); | ||
|
||
auto im = cv::imread(image_file); | ||
fastdeploy::vision::KeyPointDetectionResult res; | ||
if (!tinypose_model.Predict(&im, &res)) { | ||
std::cerr << "TinyPose Prediction Failed." << std::endl; | ||
return; | ||
} else { | ||
std::cout << "TinyPose Prediction Done!" << std::endl; | ||
} | ||
|
||
std::cout << res.Str() << std::endl; | ||
|
||
auto tinypose_vis_im = fastdeploy::vision::VisKeypointDetection(im, res, 0.5); | ||
cv::imwrite("tinypose_vis_result.jpg", tinypose_vis_im); | ||
std::cout << "TinyPose visualized result saved in ./tinypose_vis_result.jpg" | ||
<< std::endl; | ||
} | ||
|
||
int main(int argc, char* argv[]) { | ||
if (argc < 4) { | ||
std::cout << "Usage: infer_demo path/to/pptinypose_model_dir path/to/image " | ||
"run_option, " | ||
"e.g ./infer_model ./pptinypose_model_dir ./test.jpeg 0" | ||
<< std::endl; | ||
std::cout << "The data type of run_option is int, 0: run with cpu; 1: run " | ||
"with gpu; 2: run with gpu and use tensorrt backend; 3: run " | ||
"with kunlunxin." | ||
<< std::endl; | ||
return -1; | ||
} | ||
|
||
if (std::atoi(argv[3]) == 0) { | ||
RKNPU2Infer(argv[1], argv[2]); | ||
} | ||
return 0; | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
mean: | ||
- | ||
- 123.675 | ||
- 116.28 | ||
- 103.53 | ||
std: | ||
- | ||
- 58.395 | ||
- 57.12 | ||
- 57.375 | ||
model_path: ./PP_TinyPose_256x192_infer/PP_TinyPose_256x192_infer.onnx | ||
outputs_nodes: ['conv2d_441.tmp_1'] | ||
do_quantization: False | ||
dataset: | ||
output_folder: "./PP_TinyPose_256x192_infer" |