English | 简体中文
Installation | Documents | API Docs | Release Notes
⚡️FastDeploy is an Easy-to-use and High Performance AI model deployment toolkit for Cloud, Mobile and Edge with 📦out-of-the-box and unified experience, 🔚end-to-end optimization for over 🔥150+ Text, Vision, Speech and Cross-modal AI models. Including image classification, object detection, image segmentation, face detection, face recognition, keypoint detection, matting, OCR, NLP, TTS and other tasks to meet developers' industrial deployment needs for multi-scenario, multi-hardware and multi-platform.
Image Classification | Object Detection | Semantic Segmentation | Potrait Segmentation |
---|---|---|---|
Image Matting | Real-Time Matting | OCR | Face Alignment |
Pose Estimation | Behavior Recognition | NLP | Speech |
input:Life was like a box |
-
Slack:Join our Slack community and chat with other community members about ideas.
-
WeChat:Scan the QR code below using WeChat, follow the PaddlePaddle official account and fill out the questionnaire to join the WeChat group.
-
🖥️ Server-side and Cloud Deployment
-
📱 Mobile and Edge Device Deployment
-
🌐 Browser and Mini Program Deployment
A Quick Start for Python SDK(click to expand)
- CUDA >= 11.2 、cuDNN >= 8.0 、 Python >= 3.6
- OS: Linux x86_64/macOS/Windows 10
pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
pip install fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
- Prepare model and picture
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
# For deployment of GPU/TensorRT, please refer to examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision
im = cv2.imread("000000014439.jpg")
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
result = model.predict(im)
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
A Quick Start for C++ SDK(click to expand)
- Please refer to C++ Prebuilt Libraries Download
- Prepare models and pictures
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
// For GPU/TensorRT deployment, please refer to examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto im = cv::imread("000000014439.jpg");
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
vision::DetectionResult res;
model.Predict(&im, &res);
auto vis_im = vision::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
return 0;
}
For more deployment models, please refer to Vision Model Deployment Examples .
Notes: ✅: already supported; ❔: to be supported in the future; N/A: Not Available;
Server-side and Cloud Model List(click to fold)
Task | Model | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux | Linux |
---|---|---|---|---|---|---|---|---|---|---|---|---|
--- | --- | X86 CPU | NVIDIA GPU | Intel CPU | NVIDIA GPU | Intel CPU | Arm CPU | AArch64 CPU | Phytium D2000CPU | NVIDIA Jetson | Graphcore IPU | Serving |
Classification | PaddleClas/ResNet50 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | TorchVison/ResNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | ltralytics/YOLOv5Cls | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-LCNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/PP-LCNetv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/EfficientNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/GhostNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/ShuffleNetV2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/SqueeezeNetV1.1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/Inceptionv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Classification | PaddleClas/PP-HGNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/SwinTransformer | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/PP-YOLOE | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/PicoDet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/YOLOX | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/YOLOv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/PP-YOLO | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/PP-YOLOv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/Faster-RCNN | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/Mask-RCNN | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | Megvii-BaseDetection/YOLOX | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_trt | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_ort_ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | meituan/YOLOv6 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | ultralytics/YOLOv5 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Detection | WongKinYiu/YOLOR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/ScaledYOLOv4 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | ppogg/YOLOv5Lite | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | RangiLyu/NanoDetPlus | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
KeyPoint | PaddleDetection/TinyPose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
KeyPoint | PaddleDetection/PicoDet + TinyPose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
HeadPose | omasaht/headpose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Tracking | PaddleDetection/PP-Tracking | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
Segmentation | PaddleSeg/PP-LiteSeg | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegLite | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/HRNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegServer | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/Unet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/Deeplabv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
FaceDetection | biubug6/RetinaFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | Linzaer/UltraFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | deepcam-cn/YOLOv5Face | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | insightface/SCRFD | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceAlign | Hsintao/PFLD | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceAlign | Single430FaceLandmark1000 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
FaceAlign | jhb86253817/PIPNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/ArcFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/CosFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/PartialFC | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/VPL | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | ZHKKKe/MODNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Matting | PeterL1n/RobustVideoMatting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/PP-Matting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/PP-HumanMatting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/ModNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/BasicVSR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/EDVR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/PP-MSVSR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Information Extraction | PaddleNLP/UIE | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | |
NLP | PaddleNLP/ERNIE-3.0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ✅ |
Speech | PaddleSpeech/PP-TTS | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -- | ✅ |
Mobile and Edge Model List(click to fold)
Task | Model | Size (MB) | Linux | Android | Linux | Linux | Linux | Linux | TBD... |
---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | ARM CPU | ARM CPU | Rockchip-NPU RK3568/RK3588 |
Rockchip-NPU RV1109/RV1126/RK1808 |
Amlogic-NPU A311D/S905D/C308X |
NXP-NPU i.MX 8M Plus |
TBD...| |
Classification | PaddleClas/ResNet50 | 98 | ✅ | ✅ | ✅ | ||||
Classification | PaddleClas/PP-LCNet | 11.9 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/PP-LCNetv2 | 26.6 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/EfficientNet | 31.4 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/GhostNet | 20.8 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/MobileNetV1 | 17 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/MobileNetV2 | 14.2 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/MobileNetV3 | 22 | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | -- |
Classification | PaddleClas/ShuffleNetV2 | 9.2 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/SqueezeNetV1.1 | 5 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/Inceptionv3 | 95.5 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/PP-HGNet | 59 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Classification | PaddleClas/SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- |
Detection | PaddleDetection/PP-PicoDet_s_320_coco_lcnet | 4.9 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -- |
Face Detection | deepinsight/SCRFD | 2.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Keypoint Detection | PaddleDetection/PP-TinyPose | 5.5 | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ | -- |
Segmentation | PaddleSeg/PP-LiteSeg(STDC1) | 32.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/PP-HumanSeg-Lite | 0.556 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/HRNet-w18 | 38.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/PP-HumanSeg | 107.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/Unet | 53.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/Deeplabv3 | 150 | ❔ | ✅ | ✅ | ||||
OCR | PaddleOCR/PP-OCRv2 | 2.3+4.4 | ✅ | ✅ | ❔ | -- | -- | -- | -- |
OCR | PaddleOCR/PP-OCRv3 | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
Browser-based Model List(click to fold)
Task | Model | web_demo |
---|---|---|
--- | --- | Paddle.js |
Detection | FaceDetection | ✅ |
Detection | ScrewDetection | ✅ |
Segmentation | PaddleSeg/HumanSeg | ✅ |
Object Recognition | GestureRecognition | ✅ |
Object Recognition | ItemIdentification | ✅ |
OCR | PaddleOCR/PP-OCRv3 | ✅ |
We sincerely appreciate the open-sourced capabilities in EasyEdge as we adopt it for the SDK generation and download in this project.
FastDeploy is provided under the Apache-2.0.