English | 简体中文
Documents | API Docs | Release Notes
⚡️FastDeploy is an Easy-to-use and High Performance AI model deployment toolkit for Cloud, Mobile and Edge with 📦out-of-the-box and unified experience, 🔚end-to-end optimization for over 🔥150+ Text, Vision, Speech and Cross-modal AI models. Including image classification, object detection, image segmentation, face detection, face recognition, keypoint detection, matting, OCR, NLP, TTS and other tasks to meet developers' industrial deployment needs for multi-scenario, multi-hardware and multi-platform.
Image Classification | Object Detection | Semantic Segmentation | Potrait Segmentation |
---|---|---|---|
Image Matting | Real-Time Matting | OCR | Face Alignment |
Pose Estimation | Behavior Recognition | NLP | Speech |
input:Life was like a box |
- 🔥【Live Preview】2022.11.09~2022.11.10 China Standard Time, 20:30~21:30, Engineers@FastDeploy will show Using FastDeploy Efficiently for 3 days.
- Slack:Join our Slack community and chat with other community members about ideas.
- WeChat:Scan the QR code below using WeChat, follow the PaddlePaddle official account and fill out the questionnaire to join the WeChat group.
-
🔥 2022.11.15:Release FastDeploy release v0.7.0
- 🖥️ Server-side and Cloud Deployment: Support more CV models, improve deployment performance
- Support PaddleClas model service-based deployment;
- Support Stable Diffusion model deployment;
- Upgrade PaddleClas、PaddleDetection、YOLOv5 deployment code to support predict and batch_predict;
- Add the Pad function operator for the FDTensor to support Padding of the input during batch prediction;
- Add Python API to_dlpack interface for FDTensor to support copyless transfer of FDTensor between frameworks.
- 📱 Mobile and Edge Device Deployment: support new backend,support more CV model
- Support Paddle Lite TIM-VX for supporting hardware such as Rockchip RV1109,RV1126, RK1808, Amlogic A311D, etc. And provide a seamless deployment experience with other inference engines include Paddle Inference、Paddle Inference TensorRT、Paddle Lite、TensorRT、OpenVINO、ONNX Runtime、RKNPU2;
- Support Image classification model ResNet50_vd on Rockchip RV1126;
- Support Face detection model SCRFD on Rockchip RK3588, RK3568 and other hardware.
- 🖥️ Server-side and Cloud Deployment: Support more CV models, improve deployment performance
-
📖 Tutorials(click to fold)
- Install
- How to Install FastDeploy Prebuilt Libraries
- How to Build and Install FastDeploy Library on GPU Platform
- How to Build and Install FastDeploy Library on CPU Platform
- How to Build and Install FastDeploy Library on IPU Platform
- How to Build and Install FastDeploy Library on Nvidia Jetson Platform
- How to Build and Install FastDeploy Library on Android Platform
- A Quick Start - Demos
- API (To be continued)
- Performance Optimization
- Frequent Q&As
- More FastDeploy Deployment Module
- Install
- 🖥️ Server-side and Cloud Deployment
- 📱 Mobile and Edge Device Deployment
- 🌐 Browser and Mini Program Deployment
- Community
- Acknowledge
- License
A Quick Start for Python SDK(click to fold)
- CUDA >= 11.2 、cuDNN >= 8.0 、 Python >= 3.6
- OS: Linux x86_64/macOS/Windows 10
pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
pip install fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
- Prepare model and picture
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
# For deployment of GPU/TensorRT, please refer to examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision
im = cv2.imread("000000014439.jpg")
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
result = model.predict(im)
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
A Quick Start for C++ SDK(click to expand)
- Please refer to C++ Prebuilt Libraries Download
- Prepare models and pictures
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
// For GPU/TensorRT deployment, please refer to examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto im = cv::imread("000000014439.jpg");
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
vision::DetectionResult res;
model.Predict(&im, &res);
auto vis_im = vision::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
return 0;
}
For more deployment models, please refer to Vision Model Deployment Examples .
Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Available;
Task | Model | API | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux |
---|---|---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | X86 CPU | NVIDIA GPU | Intel CPU | NVIDIA GPU | Intel CPU | Arm CPU | AArch64 CPU | NVIDIA Jetson | Graphcore IPU | Serving |
Classification | PaddleClas/ResNet50 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | TorchVison/ResNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | ltralytics/YOLOv5Cls | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-LCNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/PP-LCNetv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/EfficientNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/GhostNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/MobileNetV1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/MobileNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/MobileNetV3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/ShuffleNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/SqueeezeNetV1.1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/Inceptionv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-HGNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/SwinTransformer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/PP-YOLOE | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/PicoDet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/YOLOv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/PP-YOLO | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/PP-YOLOv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/Faster-RCNN | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/Mask-RCNN | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | Megvii-BaseDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_trt | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ||
Detection | WongKinYiu/YOLOv7end2end_ort_ | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ||
Detection | meituan/YOLOv6 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | ultralytics/YOLOv5 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOR | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/ScaledYOLOv4 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | ppogg/YOLOv5Lite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | RangiLyu/NanoDetPlus | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
KeyPoint | PaddleDetection/TinyPose | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
KeyPoint | PaddleDetection/PicoDet + TinyPose | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
HeadPose | omasaht/headpose | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Tracking | PaddleDetection/PP-Tracking | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-LiteSeg | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegLite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/HRNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegServer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/Unet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/Deeplabv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | biubug6/RetinaFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | Linzaer/UltraFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | deepcam-cn/YOLOv5Face | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
FaceDetection | insightface/SCRFD | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceAlign | Hsintao/PFLD | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/ArcFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/CosFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/PartialFC | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/VPL | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | ZHKKKe/MODNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PeterL1n/RobustVideoMatting | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/PP-Matting | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/PP-HumanMatting | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/ModNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Information Extraction | PaddleNLP/UIE | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
NLP | PaddleNLP/ERNIE-3.0 | Python/C++ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ✅ |
Speech | PaddleSpeech/PP-TTS | Python/C++ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- | ✅ |
Task | Model | Size (MB) | Linux | Android | iOS | Linux | Linux | Linux | Linux | TBD... |
---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | ARM CPU | ARM CPU | ARM CPU | Rockchip-NPU RK3568/RK3588 |
Rockchip-NPU RV1109/RV1126/RK1808 |
Amlogic-NPU A311D/S905D/C308X |
NXP-NPU i.MX 8M Plus |
TBD...| |
Classification | PaddleClas/PP-LCNet | 11.9 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/PP-LCNetv2 | 26.6 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/EfficientNet | 31.4 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/GhostNet | 20.8 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/MobileNetV1 | 17 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/MobileNetV2 | 14.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/MobileNetV3 | 22 | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
Classification | PaddleClas/ShuffleNetV2 | 9.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/SqueezeNetV1.1 | 5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/Inceptionv3 | 95.5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/PP-HGNet | 59 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-PicoDet_s_320_coco | 4.1 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-PicoDet_s_320_lcnet | 4.9 | ✅ | ✅ | ❔ | ❔ | ✅ | ✅ | ✅ | -- |
Detection | PaddleDetection/CenterNet | 4.8 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/YOLOv3_MobileNetV3 | 94.6 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLO_tiny_650e_coco | 4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/SSD_MobileNetV1_300_120e_voc | 23.3 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLO_ResNet50vd | 188.5 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLOv2_ResNet50vd | 218.7 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLO_crn_l_300e_coco | 209.1 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | YOLOv5s | 29.3 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Face Detection | BlazeFace | 1.5 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Face Detection | RetinaFace | 1.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Keypoint Detection | PaddleDetection/PP-TinyPose | 5.5 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
Segmentation | PaddleSeg/PP-LiteSeg(STDC1) | 32.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/PP-HumanSeg-Lite | 0.556 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/HRNet-w18 | 38.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/PP-HumanSeg | 107.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/Unet | 53.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/Deeplabv3 | 150 | ❔ | ❔ | ❔ | ✅ | ||||
OCR | PaddleOCR/PP-OCRv1 | 2.3+4.4 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
OCR | PaddleOCR/PP-OCRv2 | 2.3+4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
OCR | PaddleOCR/PP-OCRv3 | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
OCR | PaddleOCR/PP-OCRv3-tiny | 2.4+10.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Task | Model | web_demo |
---|---|---|
--- | --- | Paddle.js |
Detection | FaceDetection | ✅ |
Detection | ScrewDetection | ✅ |
Segmentation | PaddleSeg/HumanSeg | ✅ |
Object Recognition | GestureRecognition | ✅ |
Object Recognition | ItemIdentification | ✅ |
OCR | PaddleOCR/PP-OCRv3 | ✅ |
- If you have any question or suggestion, please give us your valuable input via GitHub Issues
- Join Us👬:
- Slack:Join our Slack community and chat with other community members about ideas
- WeChat:join our WeChat community and chat with other community members about ideas
We sincerely appreciate the open-sourced capabilities in EasyEdge as we adopt it for the SDK generation and download in this project.
FastDeploy is provided under the Apache-2.0.