Skip to content
/ FastDeploy Public
forked from PaddlePaddle/FastDeploy

⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.

www.paddlepaddle.org.cn/fastdeploy

License

Apache-2.0 license
0 stars 466 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

fasby45/FastDeploy

BranchesTags

Repository files navigation

⚡️FastDeploy

⚡️FastDeploy是一款易用高效的推理部署开发套件。覆盖业界主流优质预训练模型并提供开箱即用的部署体验，包括图像分类、目标检测、图像分割、人脸检测、人脸识别、人体关键点识别、文字识别等多任务，满足开发者多场景，多硬件、多平台的便捷高效的产业级部署需求。

Human Segmentation Image Matting Semantic Segmentation Real-Time Matting
PP-OCR Behavior Recognition Object Detection TinyPose
Face Alignment 3D Object Detection Face Editing FOM

近期更新

  • 🔥 2022.8.18：发布FastDeploy release/v0.2.0
    • 服务端部署全新升级：更快的推理性能，更多的视觉模型支持
      • 发布基于x86 CPU、NVIDIA GPU的高性能推理引擎SDK，推理速度大幅提升
      • 集成Paddle Inference、ONNX Runtime、TensorRT等推理引擎并提供统一的部署体验
      • 支持YOLOv7、YOLOv6、YOLOv5、PP-YOLOE等全系列目标检测模型并提供端到端部署示例
      • 支持人脸检测、人脸识别、实时人像抠图、图像分割等40+重点模型及Demo示例
      • 支持Python和C++两种语言部署
    • 端侧部署新增瑞芯微、晶晨、恩智浦等NPU芯片部署能力

目录

服务端部署

FastDeploy Python SDK快速开始

快速安装

前置依赖
  • CUDA >= 11.2
  • cuDNN >= 8.0
  • python >= 3.8
  • OS: Linux x86_64/macOS/Windows 10
安装GPU版本
pip install numpy opencv-python fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
安装CPU版本
pip install numpy opencv-python fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

Python 推理示例

  • 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
  • 测试推理结果
# GPU/TensorRT部署参考 examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision

model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
                                 "ppyoloe_crn_l_300e_coco/model.pdiparams",
                                 "ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im.copy())
print(result)

vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)

FastDeploy C++ SDK快速开始

安装

C++ 推理示例

  • 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
  • 测试推理结果
// GPU/TensorRT部署参考 examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"

int main(int argc, char* argv[]) {
  namespace vision = fastdeploy::vision;
  auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
                                          "ppyoloe_crn_l_300e_coco/model.pdiparams",
                                          "ppyoloe_crn_l_300e_coco/infer_cfg.yml");
  auto im = cv::imread("000000014439.jpg");

  vision::DetectionResult res;
  model.Predict(&im, &res)

  auto vis_im = vision::Visualize::VisDetection(im, res, 0.5);
  cv::imwrite("vis_image.jpg", vis_im);
}

更多部署案例请参考视觉模型部署示例 .

服务端模型支持列表 🔥🔥🔥

符号说明: (1) ✅: 已经支持; (2) ❔: 未来支持; (3) ❌: 暂不支持; (4) --: 暂不考虑;
链接说明：「模型列」会跳转到模型推理Demo代码

任务场景 模型 API Linux Linux Win Win Mac Mac Linux Linux
--- --- --- X86 CPU NVIDIA GPU Intel CPU NVIDIA GPU Intel CPU Arm CPU AArch64 CPU NVIDIA Jetson
Classification PaddleClas/ResNet50 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/PP-LCNet Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/PP-LCNetv2 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/EfficientNet Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/GhostNet Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/MobileNetV1 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/MobileNetV2 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/MobileNetV3 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/ShuffleNetV2 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/SqueeezeNetV1.1 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/Inceptionv3 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/PP-HGNet Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Classification PaddleClas/SwinTransformer Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection PaddleDetection/PP-YOLOE Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection PaddleDetection/PicoDet Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection PaddleDetection/YOLOX Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection PaddleDetection/YOLOv3 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection PaddleDetection/PP-YOLO Python/C++ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❔
Detection PaddleDetection/PP-YOLOv2 Python/C++ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❔
Detection PaddleDetection/FasterRCNN Python/C++ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❔
Detection Megvii-BaseDetection/YOLOX Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection WongKinYiu/YOLOv7 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection meituan/YOLOv6 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection ultralytics/YOLOv5 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection WongKinYiu/YOLOR Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection WongKinYiu/ScaledYOLOv4 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection ppogg/YOLOv5Lite Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection RangiLyu/NanoDetPlus Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Segmentation PaddleSeg/PP-LiteSeg Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Segmentation PaddleSeg/PP-HumanSegLite Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Segmentation PaddleSeg/HRNet Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Segmentation PaddleSeg/PP-HumanSegServer Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Segmentation PaddleSeg/Unet Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Segmentation PaddleSeg/Deeplabv3 Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
FaceDetection biubug6/RetinaFace Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
FaceDetection Linzaer/UltraFace Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
FaceDetection deepcam-cn/YOLOv5Face Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
FaceDetection deepinsight/SCRFD Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
FaceRecognition deepinsight/ArcFace Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
FaceRecognition deepinsight/CosFace Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
FaceRecognition deepinsight/PartialFC Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
FaceRecognition deepinsight/VPL Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔
Matting ZHKKKe/MODNet Python/C++ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❔

端侧部署

EasyEdge边缘端部署

EasyEdge移动端部署

EasyEdge自定义模型部署

Paddle Lite NPU部署

端侧模型支持列表

任务场景 模型 大小(MB) Linux Android iOS Linux Linux Linux 更新中...
--- --- --- ARM CPU ARM CPU ARM CPU 瑞芯微NPU
RV1109
RV1126
RK1808		 晶晨NPU
A311D
S905D
C308X		 恩智浦NPU
i.MX 8M Plus		 更新中...｜
Classification PP-LCNet 11.9 ✅ ✅ ✅ -- -- -- --
Classification PP-LCNetv2 26.6 ✅ ✅ ✅ -- -- -- --
Classification EfficientNet 31.4 ✅ ✅ ✅ -- -- -- --
Classification GhostNet 20.8 ✅ ✅ ✅ -- -- -- --
Classification MobileNetV1 17 ✅ ✅ ✅ -- -- -- --
Classification MobileNetV2 14.2 ✅ ✅ ✅ -- -- -- --
Classification MobileNetV3 22 ✅ ✅ ✅ ❔ ❔ ❔ ❔
Classification ShuffleNetV2 9.2 ✅ ✅ ✅ -- -- -- --
Classification SqueezeNetV1.1 5 ✅ ✅ ✅
Classification Inceptionv3 95.5 ✅ ✅ ✅ -- -- -- --
Classification PP-HGNet 59 ✅ ✅ ✅ -- -- -- --
Classification SwinTransformer_224_win7 352.7 ✅ ✅ ✅ -- -- -- --
Detection PP-PicoDet_s_320_coco 4.1 ✅ ✅ ✅ -- -- -- --
Detection PP-PicoDet_s_320_lcnet 4.9 ✅ ✅ ✅ ✅ ✅ ✅ ❔
Detection CenterNet 4.8 ✅ ✅ ✅ -- -- -- --
Detection YOLOv3_MobileNetV3 94.6 ✅ ✅ ✅ -- -- -- --
Detection PP-YOLO_tiny_650e_coco 4.4 ✅ ✅ ✅ -- -- -- --
Detection SSD_MobileNetV1_300_120e_voc 23.3 ✅ ✅ ✅ -- -- -- --
Detection PP-YOLO_ResNet50vd 188.5 ✅ ✅ ✅ -- -- -- --
Detection PP-YOLOv2_ResNet50vd 218.7 ✅ ✅ ✅ -- -- -- --
Detection PP-YOLO_crn_l_300e_coco 209.1 ✅ ✅ ✅ -- -- -- --
Detection YOLOv5s 29.3 ✅ ✅ ✅ -- -- -- --
FaceDetection BlazeFace 1.5 ✅ ✅ ✅ -- -- -- --
FaceDetection RetinaFace 1.7 ✅ ❌ ❌ -- -- -- --
KeypointsDetection PP-TinyPose 5.5 ✅ ✅ ✅ ❔ ❔ ❔ ❔
Segmentation PP-LiteSeg(STDC1) 32.2 ✅ ✅ ✅ -- -- -- --
Segmentation PP-HumanSeg-Lite 0.556 ✅ ✅ ✅ -- -- -- --
Segmentation HRNet-w18 38.7 ✅ ✅ ✅ -- -- -- --
Segmentation PP-HumanSeg-Server 107.2 ✅ ✅ ✅ -- -- -- --
Segmentation Unet 53.7 ❌ ✅ ❌ -- -- -- --
OCR PP-OCRv1 2.3+4.4 ✅ ✅ ✅ -- -- -- --
OCR PP-OCRv2 2.3+4.4 ✅ ✅ ✅ -- -- -- --
OCR PP-OCRv3 2.4+10.6 ✅ ✅ ✅ ❔ ❔ ❔ ❔
OCR PP-OCRv3-tiny 2.4+10.7 ✅ ✅ ✅ -- -- -- --

社区交流

  • 加入社区👬： 微信扫描二维码后，填写问卷加入交流群，与开发者共同讨论推理部署痛点问题

Acknowledge

本项目中SDK生成和下载使用了EasyEdge中的免费开放能力，在此表示感谢。

License

FastDeploy遵循Apache-2.0开源协议。

About

⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.

www.paddlepaddle.org.cn/fastdeploy

Resources

Readme

License

Apache-2.0 license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • C++ 67.4%
  • Python 17.2%
  • Java 4.5%
  • CMake 2.9%
  • C# 2.7%
  • Shell 2.5%
  • Other 2.8%