English | 简体中文
⚡️FastDeploy is an accessible and efficient deployment Development Toolkit. It covers 🔥critical AI models in the industry and provides 📦out-of-the-box deployment experience. It covers image classification, object detection, image segmentation, face detection, face recognition, human keypoint detection, OCR, semantic understanding and other tasks to meet developers' industrial deployment needs for multi-scenario, multi-hardware and multi-platform .
Potrait Segmentation | Image Matting | Semantic Segmentation | Real-Time Matting |
---|---|---|---|
OCR | Behavior Recognition | Object Detection | Pose Estimation |
Face Alignment | 3D Object Detection | Face Editing | Image Animation |
-
🔥 2022.10.31:Release FastDeploy release v0.5.0
- New deployment upgrade: Support support more backend, support more cv models
- Support Paddle Inference TensorRT, and provide a seamless deployment experience with other inference engines include TensorRT、OpenVINO、ONNX Runtime、Paddle Lite、Paddle Inference;
- Support Graphcore IPU through paddle Inference;
- Support tracking model PP-Tracking and RobustVideoMatting model
- New deployment upgrade: Support support more backend, support more cv models
-
🔥 2022.10.24:Release FastDeploy release v0.4.0
- New server-side deployment upgrade: support more CV model and NLP model
- Integrate Paddle Lite and provide a seamless deployment experience with other inference engines include TensorRT、OpenVINO、ONNX Runtime、Paddle Inference;
- Support Lightweight Detection Model and classification model on Android Platform,Download to try it out
- end-to-end optimization on GPU, YOLO series model end-to-end inference speedup from 43ms to 25ms;
- Web deployment and Mini Program deployment New OCR and other CV models capability;
- Support TinyPose and PicoDet+TinyPosePipeline deployment;
- New CV models include PP-OCRv3, PP-OCRv2, PP-TinyPose, PP-Matting, etc. and provides end-to-end deployment demos
- New information extraction model is UIE, and provides end-to-end deployment demos.
- New server-side deployment upgrade: support more CV model and NLP model
- Data Center and Cloud Deployment
- Mobile and Edge Device Deployment
- Community
- Acknowledge
- License
- CUDA >= 11.2
- cuDNN >= 8.0
- python >= 3.6
- OS: Linux x86_64/macOS/Windows 10
pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
pip install fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
- Prepare models and pictures
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
# For deployment of GPU/TensorRT, please refer to examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im.copy())
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
A Quick Start for C++ SDK
- Please refer to C++ Prebuilt Libraries Download
- Prepare models and pictures
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
// For GPU/TensorRT deployment, please refer to examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
auto im = cv::imread("000000014439.jpg");
vision::DetectionResult res;
model.Predict(&im, &res);
auto vis_im = vision::Visualize::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
return 0;
}
For more deployment models, please refer to Vision Model Deployment Examples .
Notes: ✅: already supported; ❔: to be supported in the future; ❌: not supported now;
Task | Model | API | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | web_demo | mini_program |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | X86 CPU | NVIDIA GPU | Intel CPU | NVIDIA GPU | Intel CPU | Arm CPU | AArch64 CPU | NVIDIA Jetson | Graphcore IPU | Paddle.js | Paddle.js |
Classification | PaddleClas/ResNet50 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-LCNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-LCNetv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/EfficientNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/GhostNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/MobileNetV1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/MobileNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/ShuffleNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/SqueeezeNetV1.1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/Inceptionv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Classification | PaddleClas/PP-HGNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/SwinTransformer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | PaddleDetection/PP-YOLOE | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | PaddleDetection/PicoDet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | PaddleDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | PaddleDetection/YOLOv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | PaddleDetection/PP-YOLO | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❔ | ❔ |
Detection | PaddleDetection/PP-YOLOv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❔ | ❔ |
Detection | PaddleDetection/FasterRCNN | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❔ | ❔ |
Detection | PaddleDetection/PP-Tracking | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❔ | ❔ |
Detection | Megvii-BaseDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | meituan/YOLOv6 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | ultralytics/YOLOv5 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | WongKinYiu/YOLOR | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | WongKinYiu/ScaledYOLOv4 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | ppogg/YOLOv5Lite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Detection | RangiLyu/NanoDetPlus | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ |
Segmentation | PaddleSeg/PP-LiteSeg | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegLite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Segmentation | PaddleSeg/HRNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegServer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ |
Segmentation | PaddleSeg/Unet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Segmentation | PaddleSeg/Deeplabv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Face Detection | biubug6/RetinaFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Face Detection | Linzaer/UltraFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
FaceDetection | deepcam-cn/YOLOv5Face | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Face Detection | insightface/SCRFD | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Face Recognition | insightface/ArcFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Face Recognition | insightface/CosFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Face Recognition | insightface/PartialFC | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Face Recognition | insightface/VPL | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Matting | ZHKKKe/MODNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Matting | eterL1n/RobustVideoMatting | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Matting | PaddleSeg/PP-Matting | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Matting | PaddleSeg/PP-HumanMatting | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Matting | PaddleSeg/ModNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Information Extraction | PaddleNLP/UIE | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Text Classification | PaddleNLP/Ernie-3.0 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Text-to-Speech | PaddleSpeech/PP-TTS | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❔ | ❔ |
Model | Size (MB) | Linux | Android | iOS | Linux | Linux | Linux | TBD... | |
---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | ARM CPU | ARM CPU | ARM CPU | Rockchip-NPU RV1109 RV1126 RK1808 |
Amlogic-NPU A311D S905D C308X |
NXPNPU i.MX 8M Plus |
TBD...| |
Classification | PP-LCNet | 11.9 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | PP-LCNetv2 | 26.6 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | EfficientNet | 31.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | GhostNet | 20.8 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | MobileNetV1 | 17 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | MobileNetV2 | 14.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | MobileNetV3 | 22 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Classification | ShuffleNetV2 | 9.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | SqueezeNetV1.1 | 5 | ✅ | ✅ | ✅ | ||||
Classification | Inceptionv3 | 95.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | PP-HGNet | 59 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-PicoDet_s_320_coco | 4.1 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-PicoDet_s_320_lcnet | 4.9 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | CenterNet | 4.8 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | YOLOv3_MobileNetV3 | 94.6 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLO_tiny_650e_coco | 4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | SSD_MobileNetV1_300_120e_voc | 23.3 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLO_ResNet50vd | 188.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLOv2_ResNet50vd | 218.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLO_crn_l_300e_coco | 209.1 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | YOLOv5s | 29.3 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Face Detection | BlazeFace | 1.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Face Detection | RetinaFace | 1.7 | ✅ | ❌ | ❌ | -- | -- | -- | -- |
Keypoint Detection | PP-TinyPose | 5.5 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Segmentation | PP-LiteSeg(STDC1) | 32.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PP-HumanSeg-Lite | 0.556 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | HRNet-w18 | 38.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PP-HumanSeg-Server | 107.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | Unet | 53.7 | ❌ | ✅ | ❌ | -- | -- | -- | -- |
OCR | PP-OCRv1 | 2.3+4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
OCR | PP-OCRv2 | 2.3+4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
OCR | PP-OCRv3 | 2.4+10.6 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
OCR | PP-OCRv3-tiny | 2.4+10.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
- If you have any question or suggestion, please give us your valuable input via GitHub Issues
- Join Us👬:
- Slack:Join our Slack community and chat with other community members about ideas
- WeChat:join our WeChat community and chat with other community members about ideas
We sincerely appreciate the open-sourced capabilities in EasyEdge as we adopt it for the SDK generation and download in this project.
FastDeploy is provided under the Apache-2.0.