English | 简体中文
Installation | Documents | API Docs | Release Notes
⚡️FastDeploy is an Easy-to-use and High Performance AI model deployment toolkit for Cloud, Mobile and Edge with 📦out-of-the-box and unified experience, 🔚end-to-end optimization for over 🔥150+ Text, Vision, Speech and Cross-modal AI models. Including image classification, object detection, image segmentation, face detection, face recognition, keypoint detection, matting, OCR, NLP, TTS and other tasks to meet developers' industrial deployment needs for multi-scenario, multi-hardware and multi-platform.
Image Classification | Object Detection | Semantic Segmentation | Potrait Segmentation |
---|---|---|---|
Image Matting | Real-Time Matting | OCR | Face Alignment |
Pose Estimation | Behavior Recognition | NLP | Speech |
input:Life was like a box |
-
Community
-
Slack:Join our Slack community and chat with other community members about ideas.
-
WeChat:Scan the QR code below using WeChat, follow the PaddlePaddle official account and fill out the questionnaire to join the WeChat group.
-
-
🔥 2022.11.23:Release FastDeploy release v0.8.0
-
🖥️ Server-side and Cloud Deployment: Support more CV models, improve deployment performance
- Support PIPNet, FaceLandmark1000 face alignment models deployment;
- Support Video Super-Resolution series model PP-MSVSR、EDVR、BasicVSR;
- Upgrade YOLOv7 deployment code to add
batch_predict
deployment; - Support UIE service-based deployment;
- Add Python API to_dlpack interface for FDTensor to support copyless transfer of FDTensor between frameworks.
-
📱 Mobile and Edge Device Deployment: support more CV model
- Support Android image classification, target detection, semantic segmentation, OCR, face detection APK projects and examples.
Image Classification Object Detection Semantic Segmentation OCR Face Detection Project Code Project Code Project Code Project Code Project Code Scan the code
or click on the link
to installScan the code
or click on the link
to installScan the code
or click on the link
to installScan the code
or click on the link
to installScan the code
or click on the link
to install
-
-
📖 Tutorials(click to fold)
-
Install
- How to Install FastDeploy Prebuilt Libraries
- How to Build and Install FastDeploy Library on GPU Platform
- How to Build and Install FastDeploy Library on CPU Platform
- How to Build and Install FastDeploy Library on IPU Platform
- How to Build and Install FastDeploy Library on Nvidia Jetson Platform
- How to Build and Install FastDeploy Library on Android Platform
- How to Install FastDeploy Images on Serving Deployment
-
A Quick Start - Demos
-
API (To be continued)
-
Performance Optimization
-
Frequent Q&As
-
More FastDeploy Deployment Module
-
-
🖥️ Server-side and Cloud Deployment
-
📱 Mobile and Edge Device Deployment
-
🌐 Browser and Mini Program Deployment
A Quick Start for Python SDK(click to expand)
- CUDA >= 11.2 、cuDNN >= 8.0 、 Python >= 3.6
- OS: Linux x86_64/macOS/Windows 10
pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
pip install fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
- Prepare model and picture
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
# For deployment of GPU/TensorRT, please refer to examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision
im = cv2.imread("000000014439.jpg")
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
result = model.predict(im)
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
A Quick Start for C++ SDK(click to expand)
- Please refer to C++ Prebuilt Libraries Download
- Prepare models and pictures
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
// For GPU/TensorRT deployment, please refer to examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto im = cv::imread("000000014439.jpg");
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
vision::DetectionResult res;
model.Predict(&im, &res);
auto vis_im = vision::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
return 0;
}
For more deployment models, please refer to Vision Model Deployment Examples .
Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Available;
Server-side and Cloud Model List(click to fold)
Task | Model | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux |
---|---|---|---|---|---|---|---|---|---|---|---|
--- | --- | X86 CPU | NVIDIA GPU | Intel CPU | NVIDIA GPU | Intel CPU | Arm CPU | AArch64 CPU | NVIDIA Jetson | Graphcore IPU | Serving |
Classification | PaddleClas/ResNet50 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | TorchVison/ResNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | ltralytics/YOLOv5Cls | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-LCNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/PP-LCNetv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/EfficientNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/GhostNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/ShuffleNetV2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/SqueeezeNetV1.1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/Inceptionv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Classification | PaddleClas/PP-HGNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/SwinTransformer | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/PP-YOLOE | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/PicoDet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/YOLOX | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/YOLOv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/PP-YOLO | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/PP-YOLOv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/Faster-RCNN | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/Mask-RCNN | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | Megvii-BaseDetection/YOLOX | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_trt | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_ort_ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | meituan/YOLOv6 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | ultralytics/YOLOv5 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/ScaledYOLOv4 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | ppogg/YOLOv5Lite | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | RangiLyu/NanoDetPlus | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
KeyPoint | PaddleDetection/TinyPose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
KeyPoint | PaddleDetection/PicoDet + TinyPose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
HeadPose | omasaht/headpose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Tracking | PaddleDetection/PP-Tracking | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
OCR | PaddleOCR/PP-OCRv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-LiteSeg | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegLite | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/HRNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegServer | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/Unet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/Deeplabv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | biubug6/RetinaFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | Linzaer/UltraFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | deepcam-cn/YOLOv5Face | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
FaceDetection | insightface/SCRFD | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceAlign | Hsintao/PFLD | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceAlign | Single430FaceLandmark1000 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceAlign | jhb86253817/PIPNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/ArcFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/CosFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/PartialFC | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/VPL | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | ZHKKKe/MODNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PeterL1n/RobustVideoMatting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/PP-Matting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/PP-HumanMatting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/ModNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/BasicVSR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/EDVR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/PP-MSVSR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Information Extraction | PaddleNLP/UIE | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
NLP | PaddleNLP/ERNIE-3.0 | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ✅ |
Speech | PaddleSpeech/PP-TTS | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | -- | ✅ |
Mobile and Edge Model List(click to fold)
Task | Model | Size (MB) | Linux | Android | iOS | Linux | Linux | Linux | Linux | TBD... |
---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | ARM CPU | ARM CPU | ARM CPU | Rockchip-NPU RK3568/RK3588 |
Rockchip-NPU RV1109/RV1126/RK1808 |
Amlogic-NPU A311D/S905D/C308X |
NXP-NPU i.MX 8M Plus |
TBD...| |
Classification | PaddleClas/PP-LCNet | 11.9 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/PP-LCNetv2 | 26.6 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/EfficientNet | 31.4 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/GhostNet | 20.8 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/MobileNetV1 | 17 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/MobileNetV2 | 14.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/MobileNetV3 | 22 | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
Classification | PaddleClas/ShuffleNetV2 | 9.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/SqueezeNetV1.1 | 5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/Inceptionv3 | 95.5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/PP-HGNet | 59 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-PicoDet_s_320_coco | 4.1 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-PicoDet_s_320_lcnet | 4.9 | ✅ | ✅ | ❔ | ❔ | ✅ | ✅ | ✅ | -- |
Detection | PaddleDetection/CenterNet | 4.8 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/YOLOv3_MobileNetV3 | 94.6 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLO_tiny_650e_coco | 4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/SSD_MobileNetV1_300_120e_voc | 23.3 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLO_ResNet50vd | 188.5 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLOv2_ResNet50vd | 218.7 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLO_crn_l_300e_coco | 209.1 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | YOLOv5s | 29.3 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Face Detection | BlazeFace | 1.5 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Face Detection | RetinaFace | 1.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Keypoint Detection | PaddleDetection/PP-TinyPose | 5.5 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
Segmentation | PaddleSeg/PP-LiteSeg(STDC1) | 32.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/PP-HumanSeg-Lite | 0.556 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/HRNet-w18 | 38.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/PP-HumanSeg | 107.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/Unet | 53.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/Deeplabv3 | 150 | ❔ | ❔ | ❔ | ✅ | ||||
OCR | PaddleOCR/PP-OCRv1 | 2.3+4.4 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
OCR | PaddleOCR/PP-OCRv2 | 2.3+4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
OCR | PaddleOCR/PP-OCRv3 | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
OCR | PaddleOCR/PP-OCRv3-tiny | 2.4+10.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Browser-based Model List(click to fold)
Task | Model | web_demo |
---|---|---|
--- | --- | Paddle.js |
Detection | FaceDetection | ✅ |
Detection | ScrewDetection | ✅ |
Segmentation | PaddleSeg/HumanSeg | ✅ |
Object Recognition | GestureRecognition | ✅ |
Object Recognition | ItemIdentification | ✅ |
OCR | PaddleOCR/PP-OCRv3 | ✅ |
- If you have any question or suggestion, please give us your valuable input via GitHub Issues
- Join Us👬:
- Slack:Join our Slack community and chat with other community members about ideas
- WeChat:join our WeChat community and chat with other community members about ideas
We sincerely appreciate the open-sourced capabilities in EasyEdge as we adopt it for the SDK generation and download in this project.
FastDeploy is provided under the Apache-2.0.