English | 简体中文 | हिन्दी | 日本語 | 한국인 | Pу́сский язы́к
安装 | 使用文档 | 快速开始 | API文档 | 更新日志
⚡️FastDeploy是一款全场景、易用灵活、极致高效的AI推理部署工具, 支持云边端部署。提供超过 🔥160+ Text,Vision, Speech和跨模态模型📦开箱即用的部署体验,并实现🔚端到端的推理性能优化。包括 物体检测、字符识别(OCR)、人脸、人像扣图、多目标跟踪系统、NLP、Stable Diffusion文图生成、TTS 等几十种任务场景,满足开发者多场景、多硬件、多平台的产业部署需求。
-
✨✨✨ 2023.01.17 发布 YOLOv8 在FastDeploy系列硬件的部署支持。 其中包括 Paddle YOLOv8 以及 社区 ultralytics YOLOv8
- Paddle YOLOv8 可以部署的硬件:Intel CPU、NVIDIA GPU、Jetson、飞腾、昆仑芯、昇腾、ARM CPU、RK3588 和 Sophgo TPU, 部分硬件包含 Python 部署和 C++ 部署;
- 社区 ultralytics YOLOv8 可以部署的硬件:Intel CPU、NVIDIA GPU、Jetson,均包含 Python 部署和 C++ 部署;
- FastDeploy 一行模型API切换,可以实现YOLOv8、 PP-YOLOE+、YOLOv5 等模型性能对比。
-
服务化部署结合VisualDL新增支持可视化部署。在FastDeploy容器中启动VDL服务后,即可在VDL界面修改模型配置、启动/管理模型服务、查看性能数据、发送请求等,详细操作可参考相关文档
-
✨👥✨ 社区交流
-
Slack:Join our Slack community and chat with other community members about ideas
-
微信:扫描二维码,填写问卷加入技术社区,与社区开发者交流部署产业落地痛点问题
-
X86_64 CPU | |||||||
NVDIA GPU | |||||||
飞腾 CPU | |||||||
昆仑芯 XPU | |||||||
华为昇腾 NPU | |||||||
Graphcore IPU | |||||||
算能 | |||||||
Intel 显卡 | |||||||
Jetson | |||||||
ARM CPU | |||||||
RK3588等 | |||||||
RV1126等 | |||||||
晶晨 | |||||||
恩智浦 |
- ✴️ Python SDK快速开始
- ✴️ C++ SDK快速开始
- 安装文档
- 快速使用
- 不同后端使用
- 服务化部署
- API文档
- 性能调优
- 常见问题
- 更多FastDeploy部署模块
- 模型支持列表
- 💕开发者贡献
Python SDK快速开始(点开收缩)
- CUDA >= 11.2、cuDNN >= 8.0、Python >= 3.6
- OS: Linux x86_64/macOS/Windows 10
pip install numpy opencv-python fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
pip install numpy opencv-python fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
- 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- 测试推理结果
# GPU/TensorRT部署参考 examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im)
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
C++ SDK快速开始(点开查看详情)
- 参考C++预编译库下载文档
- 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- 测试推理结果
// GPU/TensorRT部署参考 examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
auto im = cv::imread("000000014439.jpg");
vision::DetectionResult res;
model.Predict(im, &res);
auto vis_im = vision::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
return 0;
}
更多部署案例请参考模型部署示例 .
符号说明: (1) ✅ : 已经支持; (2) ❔: 正在进行中; (3) N/A : 暂不支持.
服务端模型支持列表(点击可收缩)
任务场景 | 模型 | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux | Linux | Linux | Linux | Linux | Linux |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
--- | --- | X86 CPU | NVIDIA GPU | X86 CPU | NVIDIA GPU | X86 CPU | Arm CPU | AArch64 CPU | 飞腾D2000 aarch64 | NVIDIA Jetson | Graphcore IPU | 昆仑芯 XPU | 华为 昇腾 | Serving |
Classification | PaddleClas/ResNet50 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | TorchVison/ResNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Classification | ultralytics/YOLOv5Cls | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
Classification | PaddleClas/PP-LCNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/PP-LCNetv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/EfficientNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/GhostNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/ShuffleNetV2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/SqueeezeNetV1.1 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/Inceptionv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Classification | PaddleClas/PP-HGNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | 🔥🔥PaddleDetection/PP-YOLOE+ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | 🔥PaddleDetection/YOLOv8 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Detection | 🔥ultralytics/YOLOv8 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Detection | PaddleDetection/PicoDet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/YOLOX | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/YOLOv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/PP-YOLO | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/PP-YOLOv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/Faster-RCNN | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | PaddleDetection/Mask-RCNN | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ |
Detection | Megvii-BaseDetection/YOLOX | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Detection | WongKinYiu/YOLOv7 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_trt | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Detection | WongKinYiu/YOLOv7end2end_ort | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Detection | meituan/YOLOv6 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ |
Detection | ultralytics/YOLOv5 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Detection | WongKinYiu/YOLOR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ✅ | ❔ |
Detection | WongKinYiu/ScaledYOLOv4 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Detection | ppogg/YOLOv5Lite | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ? | ❔ | ❔ | ❔ |
Detection | RangiLyu/NanoDetPlus | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
KeyPoint | PaddleDetection/TinyPose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
KeyPoint | PaddleDetection/PicoDet + TinyPose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ |
HeadPose | omasaht/headpose | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Tracking | PaddleDetection/PP-Tracking | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
OCR | PaddleOCR/PP-OCRv2 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
OCR | PaddleOCR/PP-OCRv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ |
Segmentation | PaddleSeg/PP-LiteSeg | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ❔ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegLite | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/HRNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/PP-HumanSegServer | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/Unet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ✅ | ✅ | ❔ |
Segmentation | PaddleSeg/Deeplabv3 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ✅ | ✅ | ❔ |
FaceDetection | biubug6/RetinaFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceDetection | Linzaer/UltraFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceDetection | deepcam-cn/YOLOv5Face | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceDetection | insightface/SCRFD | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceAlign | Hsintao/PFLD | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceAlign | Single430/FaceLandmark1000 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceAlign | jhb86253817/PIPNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceRecognition | insightface/ArcFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceRecognition | insightface/CosFace | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceRecognition | insightface/PartialFC | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
FaceRecognition | insightface/VPL | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Matting | ZHKKKe/MODNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Matting | PeterL1n/RobustVideoMatting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Matting | PaddleSeg/PP-Matting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Matting | PaddleSeg/PP-HumanMatting | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ✅ | ❔ |
Matting | PaddleSeg/ModNet | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/BasicVSR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/EDVR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Video Super-Resolution | PaddleGAN/PP-MSVSR | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | ❔ |
Information Extraction | PaddleNLP/UIE | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | ❔ | |
NLP | PaddleNLP/ERNIE-3.0 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ✅ | ❔ | ✅ |
Speech | PaddleSpeech/PP-TTS | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ | ❔ | -- | ❔ | ❔ | ✅ |
端侧模型支持列表(点击可收缩)
任务场景 | 模型 | 大小(MB) | Linux | Android | Linux | Linux | Linux | Linux | Linux | 更新中... |
---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | ARM CPU | ARM CPU | 瑞芯微NPU RK3588/RK3568/RK3566 |
瑞芯微NPU RV1109/RV1126/RK1808 |
晶晨NPU A311D/S905D/C308X |
恩智浦NPU i.MX 8M Plus |
更新中... | |
Classification | PaddleClas/ResNet50 | 98 | ✅ | ✅ | ✅ | ✅ | ||||
Classification | PaddleClas/PP-LCNet | 11.9 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/PP-LCNetv2 | 26.6 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/EfficientNet | 31.4 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/GhostNet | 20.8 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/MobileNetV1 | 17 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/MobileNetV2 | 14.2 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/MobileNetV3 | 22 | ✅ | ✅ | ❔ | ✅ | ❔ | ❔ | -- | |
Classification | PaddleClas/ShuffleNetV2 | 9.2 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/SqueezeNetV1.1 | 5 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/Inceptionv3 | 95.5 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Classification | PaddleClas/PP-HGNet | 59 | ✅ | ✅ | ❔ | ✅ | -- | -- | -- | |
Detection | PaddleDetection/PicoDet_s | 4.9 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -- | |
Detection | YOLOv5 | ❔ | ❔ | ✅ | ❔ | ❔ | ❔ | -- | ||
Face Detection | deepinsight/SCRFD | 2.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Keypoint Detection | PaddleDetection/PP-TinyPose | 5.5 | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ | -- | |
Segmentation | PaddleSeg/PP-LiteSeg(STDC1) | 32.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/PP-HumanSeg-Lite | 0.556 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/HRNet-w18 | 38.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/PP-HumanSeg | 107.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/Unet | 53.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- | |
Segmentation | PaddleSeg/Deeplabv3 | 150 | ❔ | ✅ | ✅ | |||||
OCR | PaddleOCR/PP-OCRv2 | 2.3+4.4 | ✅ | ✅ | ❔ | -- | -- | -- | -- | |
OCR | PaddleOCR/PP-OCRv3 | 2.4+10.6 | ✅ | ❔ | ❔ | ❔ | ❔ | ❔ | -- |
Web和小程序部署支持列表(点击可收缩)
任务场景 | 模型 | web_demo |
---|---|---|
--- | --- | Paddle.js |
Detection | FaceDetection | ✅ |
Detection | ScrewDetection | ✅ |
Segmentation | PaddleSeg/HumanSeg | ✅ |
Object Recognition | GestureRecognition | ✅ |
Object Recognition | ItemIdentification | ✅ |
OCR | PaddleOCR/PP-OCRv3 | ✅ |
本项目中SDK生成和下载使用了EasyEdge中的免费开放能力,在此表示感谢。
FastDeploy遵循Apache-2.0开源协议。