English | 简体中文
⚡️FastDeploy is an accessible and efficient deployment Development Toolkit. It covers 🔥hot AI models in the industry and provides 📦out-of-the-box deployment experience. It covers image classification, object detection, image segmentation, face detection, face recognition, human keypoint detection, OCR, semantic understanding and other tasks to meet developers‘ industrial deployment needs for multi-scenario, multi-hardware, multi-platform .
Potrait Segmentation | Image Matting | Semantic Segmentation | Real-Time Matting |
---|---|---|---|
OCR | Behavior Recognition | Object Detection | Pose Estimation |
Face Alignment | 3D Object Detection | Face Editing | Image Animation |
- 🔥 2022.8.18:Release FastDeploy release/v0.2.0
- New server-side deployment upgrade: faster inference performance, support more visual model
- Release high-performance inference engine SDK based on x86 CPUs and NVIDIA GPUs, with significant increase in inference speed
- Integrate Paddle Inference, ONNXRuntime, TensorRT and other inference engines and provide a seamless deployment experience
- Supports full range of object detection models such as YOLOv7, YOLOv6, YOLOv5, PP-YOLOE and provides [End-To-End Deployment Demos]](examples/vision/detection/)
- Support over 40 key models and Demo Examples including face detection, face recognition, real-time portrait matting, image segmentation.
- Support deployment in both Python and C++
- Supports Rexchip, Amlogic, NXP and other NPU chip deployment capabilities on end-side deployment
- Release Lightweight Object Detection [Picodet-NPU Deployment Demo](https://github.com/PaddlePaddle/Paddle-Lite-Demo/tree/develop/object_detection/linux/picodet_ detection), providing the full quantized inference capability for INT8.
- New server-side deployment upgrade: faster inference performance, support more visual model
- Server-side deployment
- End-side deployment
- Community
- Acknowledge
- License
- CUDA >= 11.2
- cuDNN >= 8.0
- python >= 3.8
- OS: Linux x86_64/macOS/Windows 10
pip install numpy opencv-python fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
pip install numpy opencv-python fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
- Prepare models and pictures
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
-
Test inference resultsTest inference results
# For deployment of GPU/TensorRT, please refer to examples/vision/detection/paddledetection/python import cv2 import fastdeploy.vision as vision
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel", "ppyoloe_crn_l_300e_coco/model.pdiparams", "ppyoloe_crn_l_300e_coco/infer_cfg.yml") im = cv2.imread("000000014439.jpg") result = model.predict(im.copy()) print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5) cv2.imwrite("vis_image.jpg", vis_im)
- Please refer to C++ Prebuilt Libraries Download
- Prepare models and pictures
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
// For GPU/TensorRT deployment, please refer to examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
auto im = cv::imread("000000014439.jpg");
vision::DetectionResult res;
model.Predict(&im, &res)
auto vis_im = vision::Visualize::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
For more deployment models, please refer to Visual Model Deployment Examples .
Notes:
(1) ✅: already supported; (2) ❔: to be supported in the future; (3) ❌: not supported at the moment; (4) --: not considered at the moment;
Hyperlinks:Click model's name, the website will jump to the model inference demo code
Task | Model | API | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux |
---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | X86 CPU | NVIDIA GPU | Intel CPU | NVIDIA GPU | Intel CPU | Arm CPU | AArch64 CPU | NVIDIA Jetson |
Classification | PaddleClas/ResNet50 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/PP-LCNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/PP-LCNetv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/EfficientNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/GhostNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/MobileNetV1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/MobileNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/MobileNetV3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/ShuffleNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/SqueeezeNetV1.1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/Inceptionv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/PP-HGNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/SwinTransformer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | PaddleDetection/PP-YOLOE | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | PaddleDetection/PicoDet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | PaddleDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | PaddleDetection/YOLOv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | PaddleDetection/PP-YOLO | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❔ |
Detection | PaddleDetection/PP-YOLOv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❔ |
Detection | PaddleDetection/FasterRCNN | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❔ |
Detection | Megvii-BaseDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | WongKinYiu/YOLOv7 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | meituan/YOLOv6 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | ultralytics/YOLOv5 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | WongKinYiu/YOLOR | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | WongKinYiu/ScaledYOLOv4 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | ppogg/YOLOv5Lite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | RangiLyu/NanoDetPlus | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/PP-LiteSeg | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegLite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/HRNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegServer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/Unet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/Deeplabv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
FaceDetection | biubug6/RetinaFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
FaceDetection | Linzaer/UltraFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
FaceDetection | deepcam-cn/YOLOv5Face | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
FaceDetection | deepinsight/SCRFD | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
FaceRecognition | deepinsight/ArcFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
FaceRecognition | deepinsight/CosFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
FaceRecognition | deepinsight/PartialFC | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
FaceRecognition | deepinsight/VPL | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Matting | ZHKKKe/MODNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
- ARM Linux System
Model | Size (MB) | Linux | Android | iOS | Linux | Linux | Linux | TBD... | |
---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | ARM CPU | ARM CPU | ARM CPU | Rexchip-NPU RV1109 RV1126 RK1808 |
Amlogic-NPU A311D S905D C308X |
NXPNPU i.MX 8M Plus |
TBD...| |
Classification | PP-LCNet | 11.9 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | PP-LCNetv2 | 26.6 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | EfficientNet | 31.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | GhostNet | 20.8 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | MobileNetV1 | 17 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | MobileNetV2 | 14.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | MobileNetV3 | 22 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Classification | ShuffleNetV2 | 9.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | SqueezeNetV1.1 | 5 | ✅ | ✅ | ✅ | ||||
Classification | Inceptionv3 | 95.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | PP-HGNet | 59 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-PicoDet_s_320_coco | 4.1 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-PicoDet_s_320_lcnet | 4.9 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | CenterNet | 4.8 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | YOLOv3_MobileNetV3 | 94.6 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLO_tiny_650e_coco | 4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | SSD_MobileNetV1_300_120e_voc | 23.3 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLO_ResNet50vd | 188.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLOv2_ResNet50vd | 218.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLO_crn_l_300e_coco | 209.1 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | YOLOv5s | 29.3 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
FaceDetection | BlazeFace | 1.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
FaceDetection | RetinaFace | 1.7 | ✅ | ❌ | ❌ | -- | -- | -- | -- |
KeypointsDetection | PP-TinyPose | 5.5 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Segmentation | PP-LiteSeg(STDC1) | 32.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PP-HumanSeg-Lite | 0.556 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | HRNet-w18 | 38.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PP-HumanSeg-Server | 107.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | Unet | 53.7 | ❌ | ✅ | ❌ | -- | -- | -- | -- |
OCR | PP-OCRv1 | 2.3+4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
OCR | PP-OCRv2 | 2.3+4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
OCR | PP-OCRv3 | 2.4+10.6 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
OCR | PP-OCRv3-tiny | 2.4+10.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
- If you have any question or suggestion, please give us your valuable input via GitHub Issues
- Join Us👬: Scan the QR code via WeChat to join our FastDeploy technology communication group (you )
We sincerely appreciate the open-sourced capabilities in EasyEdge as we adopt it for the SDK generation and download in this project.
FastDeploy is provided under Apache-2.0.