English | 简体中文
⚡️FastDeploy是一款易用高效的推理部署开发套件。覆盖业界🔥热门CV、NLP、Speech的AI模型并提供📦开箱即用的部署体验,包括图像分类、目标检测、图像分割、人脸检测、人脸识别、人体关键点识别、文字识别、语义理解等多任务,满足开发者多场景,多硬件、多平台的产业部署需求。
Object Detection | 3D Object Detection | Semantic Segmentation | Potrait Segmentation |
---|---|---|---|
Image Matting | Real-Time Matting | OCR | Face Alignment |
Pose Estimation | Behavior Recognition | NLP | Speech |
input :早上好,今天是2020 |
- 🔥 【直播分享】2022.11.09 20:30~21:30,《覆盖云边端全场景,150+热门模型快速部署》。微信扫码报名
- 🔥 【直播分享】2022.11.10 20:30~21:30,《瑞芯微、晶晨、恩智浦等10+AI硬件部署,直达产业落地》。微信扫码报名
- 🔥 【直播分享】2022.11.10 19:00~20:00,《10+热门模型在RK3588、RK3568部署实战》。微信扫码报名
-
🔥 2022.10.31:Release FastDeploy release v0.5.0
- 🖥️ 服务端部署:支持推理速度更快的后端,支持更多的模型
- 集成 Paddle Inference TensorRT后端,并保证其使用与Paddle Inference、TensorRT、OpenVINO、ONNX Runtime、Paddle Lite等一致的开发体验;
- 支持并测试 Graphcore IPU 通过 Paddle Inference后端;
- 优化一键模型量化工具,支持YOLOv7、YOLOv6、YOLOv5等视觉模型,在CPU和GPU推理速度可提升1.5~2倍;
- 新增 PP-Tracking 和 RobustVideoMatting 等模型;
- 🖥️ 服务端部署:支持推理速度更快的后端,支持更多的模型
-
🔥 2022.10.24:Release FastDeploy release v0.4.0
- 🖥️ 服务端部署:推理速度大升级
- 升级 GPU 端到端的优化,在YOLO系列上,模型推理速度从 43ms 提升到 25ms;
- 新增 TinyPose and PicoDetji lianTinyPosePipeline部署能力;
- 📲 移动端和端侧部署:移动端后端能力升级,支持更多的CV模型
- Web和小程序部署:新增Web端部署能力
- 集成 Paddle.js部署能力,新增 OCR、目标检测、人像分割背景替换、物体识别等Web端部署能力和Demo;
- 🖥️ 服务端部署:推理速度大升级
-
📖 文档教程(点击可收缩)
- 安装文档
- 快速使用
- API文档(进行中)
- 性能调优
- 常见问题
- 更多FastDeploy部署模块
- 🖥️ 服务器端部署
- 📲 移动端和端侧部署
- Web和小程序部署
- 社区交流
- Acknowledge
- License
Python SDK快速开始(点击可收缩)
- CUDA >= 11.2、cuDNN >= 8.0、Python >= 3.6
- OS: Linux x86_64/macOS/Windows 10
pip install numpy opencv-python fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
pip install numpy opencv-python fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
- 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- 测试推理结果
# GPU/TensorRT部署参考 examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im.copy())
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
C++ SDK快速开始(点开查看详情)
- 参考C++预编译库下载文档
- 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- 测试推理结果
// GPU/TensorRT部署参考 examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
auto im = cv::imread("000000014439.jpg");
vision::DetectionResult res;
model.Predict(&im, &res);
auto vis_im = vision::Visualize::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
return 0;
}
更多部署案例请参考视觉模型部署示例 .
符号说明: (1) ✅: 已经支持; (2) ❔: 正在进行中; (3) N/A: 暂不支持;
链接说明:「模型列」会跳转到模型推理Demo代码
任务场景 | 模型 | API | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux |
---|---|---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | X86 CPU | NVIDIA GPU | X86 CPU | NVIDIA GPU | X86 CPU | Arm CPU | AArch64 CPU | NVIDIA Jetson | Graphcore IPU | Serving |
Classification | PaddleClas/ResNet50 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | TorchVison/ResNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | ltralytics/YOLOv5Cls | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-LCNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/PP-LCNetv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/EfficientNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/GhostNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/MobileNetV1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/MobileNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/MobileNetV3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/ShuffleNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/SqueeezeNetV1.1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/Inceptionv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-HGNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Classification | PaddleClas/SwinTransformer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/PP-YOLOE | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/PicoDet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/YOLOv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/PP-YOLO | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/PP-YOLOv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/Faster-RCNN | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | PaddleDetection/Mask-RCNN | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | Megvii-BaseDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_trt | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ||
Detection | WongKinYiu/YOLOv7end2end_ort_ | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ||
Detection | meituan/YOLOv6 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | ultralytics/YOLOv5 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/YOLOR | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | WongKinYiu/ScaledYOLOv4 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | ppogg/YOLOv5Lite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Detection | RangiLyu/NanoDetPlus | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
KeyPoint | PaddleDetection/TinyPose | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
KeyPoint | PaddleDetection/PicoDet + TinyPose | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
HeadPose | omasaht/headpose | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Tracking | PaddleDetection/PP-Tracking | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-LiteSeg | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegLite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/HRNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegServer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/Unet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/Deeplabv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | biubug6/RetinaFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | Linzaer/UltraFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceDetection | deepcam-cn/YOLOv5Face | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ |
FaceDetection | insightface/SCRFD | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceAlign | Hsintao/PFLD | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/ArcFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/CosFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/PartialFC | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
FaceRecognition | insightface/VPL | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | ZHKKKe/MODNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PeterL1n/RobustVideoMatting | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/PP-Matting | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/PP-HumanMatting | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Matting | PaddleSeg/ModNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
Information Extraction | PaddleNLP/UIE | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ |
NLP | PaddleNLP/ERNIE-3.0 | Python/C++ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ✅ |
Speech | PaddleSpeech/PP-TTS | Python/C++ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- | ✅ |
任务场景 | 模型 | 大小(MB) | Linux | Android | iOS | Linux | Linux | Linux | Linux | 更新中... |
---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | ARM CPU | ARM CPU | ARM CPU | 瑞芯微NPU RK3568/RK3588 |
瑞芯微NPU RV1109/RV1126/RK1808 |
晶晨NPU A311D/S905D/C308X |
恩智浦NPU i.MX 8M Plus |
更新中...| |
Classification | PaddleClas/PP-LCNet | 11.9 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/PP-LCNetv2 | 26.6 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/EfficientNet | 31.4 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/GhostNet | 20.8 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/MobileNetV1 | 17 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/MobileNetV2 | 14.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/MobileNetV3 | 22 | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
Classification | PaddleClas/ShuffleNetV2 | 9.2 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/SqueezeNetV1.1 | 5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/Inceptionv3 | 95.5 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/PP-HGNet | 59 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Classification | PaddleClas/SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-PicoDet_s_320_coco | 4.1 | ✅ | ✅ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-PicoDet_s_320_lcnet | 4.9 | ✅ | ✅ | ❔ | ❔ | ✅ | ✅ | ✅ | -- |
Detection | PaddleDetection/CenterNet | 4.8 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/YOLOv3_MobileNetV3 | 94.6 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLO_tiny_650e_coco | 4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/SSD_MobileNetV1_300_120e_voc | 23.3 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLO_ResNet50vd | 188.5 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLOv2_ResNet50vd | 218.7 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | PaddleDetection/PP-YOLO_crn_l_300e_coco | 209.1 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Detection | YOLOv5s | 29.3 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Face Detection | BlazeFace | 1.5 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Face Detection | RetinaFace | 1.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
Keypoint Detection | PaddleDetection/PP-TinyPose | 5.5 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
Segmentation | PaddleSeg/PP-LiteSeg(STDC1) | 32.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/PP-HumanSeg-Lite | 0.556 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/HRNet-w18 | 38.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/PP-HumanSeg | 107.2 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/Unet | 53.7 | ✅ | ❔ | ❔ | ✅ | -- | -- | -- | -- |
Segmentation | PaddleSeg/Deeplabv3 | 150 | ❔ | ❔ | ❔ | ✅ | ||||
OCR | PaddleOCR/PP-OCRv1 | 2.3+4.4 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
OCR | PaddleOCR/PP-OCRv2 | 2.3+4.4 | ✅ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
OCR | PaddleOCR/PP-OCRv3 | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
OCR | PaddleOCR/PP-OCRv3-tiny | 2.4+10.7 | ❔ | ❔ | ❔ | ❔ | -- | -- | -- | -- |
任务场景 | 模型 | web_demo |
---|---|---|
--- | --- | Paddle.js |
Detection | FaceDetection | ✅ |
Detection | ScrewDetection | ✅ |
Segmentation | PaddleSeg/HumanSeg | ✅ |
Object Recognition | GestureRecognition | ✅ |
Object Recognition | ItemIdentification | ✅ |
OCR | PaddleOCR/PP-OCRv3 | ✅ |
- 加入社区👬: 微信扫描二维码,进入FastDeploy技术交流群
本项目中SDK生成和下载使用了EasyEdge中的免费开放能力,在此表示感谢。
FastDeploy遵循Apache-2.0开源协议。