Skip to content

Latest commit

 

History

History
executable file
·
345 lines (257 loc) · 54.7 KB

README_CN.md

File metadata and controls

executable file
·
345 lines (257 loc) · 54.7 KB

English | 简体中文 | हिन्दी | 日本語 | 한국인 | Pу́сский язы́к

⚡️FastDeploy

安装 | 使用文档 | API文档 | 更新日志

⚡️FastDeploy是一款全场景易用灵活极致高效的AI推理部署工具。提供📦开箱即用云边端部署体验, 支持超过 🔥150+ Text, Vision, Speech跨模态模型,并实现🔚端到端的推理性能优化。包括图像分类、物体检测、图像分割、人脸检测、人脸识别、关键点检测、抠图、OCR、NLP、TTS等任务,满足开发者多场景、多硬件、多平台的产业部署需求。

Image Classification Object Detection Semantic Segmentation Potrait Segmentation
Image Matting Real-Time Matting OCR Face Alignment
Pose Estimation Behavior Recognition NLP Speech

input :早上好今天是2020
/10/29,最低温度是-3°C。

output:

📣 最新消息

  • 💥直播预告:2022.12.12 ~ 2022.12.30日每晚20:30,联合十家硬件厂商伙伴,推出部署月《产业级AI模型部署全攻略》。微信扫描下方二维码,关注公众号并填写问卷后进入官方交流群,与行业精英共同探讨AI部署话题。

目录

🖥️ 服务端部署

Python SDK快速开始(点开查看详情)

快速安装

前置依赖
  • CUDA >= 11.2、cuDNN >= 8.0、Python >= 3.6
  • OS: Linux x86_64/macOS/Windows 10
安装GPU版本
pip install numpy opencv-python fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
安装CPU版本
pip install numpy opencv-python fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html

Python 推理示例

  • 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
  • 测试推理结果
# GPU/TensorRT部署参考 examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision

model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
                                 "ppyoloe_crn_l_300e_coco/model.pdiparams",
                                 "ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im)
print(result)

vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
C++ SDK快速开始(点开查看详情)

安装

C++ 推理示例

  • 准备模型和图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
  • 测试推理结果
// GPU/TensorRT部署参考 examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"

int main(int argc, char* argv[]) {
  namespace vision = fastdeploy::vision;
  auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
                                          "ppyoloe_crn_l_300e_coco/model.pdiparams",
                                          "ppyoloe_crn_l_300e_coco/infer_cfg.yml");
  auto im = cv::imread("000000014439.jpg");

  vision::DetectionResult res;
  model.Predict(im, &res);

  auto vis_im = vision::VisDetection(im, res, 0.5);
  cv::imwrite("vis_image.jpg", vis_im);
  return 0;
}

更多部署案例请参考模型部署示例 .

服务端模型支持列表 🔥🔥🔥🔥🔥

符号说明: (1) ✅: 已经支持; (2) ❔: 正在进行中; (3) N/A: 暂不支持;

服务端模型支持列表(点击可收缩)
任务场景 模型 Linux Linux Win Win Mac Mac Linux Linux Linux Linux Linux Linux Linux
--- --- X86 CPU NVIDIA GPU X86 CPU NVIDIA GPU X86 CPU Arm CPU AArch64 CPU 飞腾D2000 aarch64 NVIDIA Jetson Graphcore IPU 昆仑芯 XPU 华为 昇腾 Serving
Classification PaddleClas/ResNet50
Classification TorchVison/ResNet
Classification ltralytics/YOLOv5Cls
Classification PaddleClas/PP-LCNet
Classification PaddleClas/PP-LCNetv2
Classification PaddleClas/EfficientNet
Classification PaddleClas/GhostNet
Classification PaddleClas/MobileNetV1
Classification PaddleClas/MobileNetV2
Classification PaddleClas/MobileNetV3
Classification PaddleClas/ShuffleNetV2
Classification PaddleClas/SqueeezeNetV1.1
Classification PaddleClas/Inceptionv3
Classification PaddleClas/PP-HGNet
Detection PaddleDetection/PP-YOLOE
Detection PaddleDetection/PicoDet
Detection PaddleDetection/YOLOX
Detection PaddleDetection/YOLOv3
Detection PaddleDetection/PP-YOLO
Detection PaddleDetection/PP-YOLOv2
Detection PaddleDetection/Faster-RCNN
Detection PaddleDetection/Mask-RCNN
Detection Megvii-BaseDetection/YOLOX
Detection WongKinYiu/YOLOv7
Detection WongKinYiu/YOLOv7end2end_trt
Detection WongKinYiu/YOLOv7end2end_ort_
Detection meituan/YOLOv6
Detection ultralytics/YOLOv5
Detection WongKinYiu/YOLOR
Detection WongKinYiu/ScaledYOLOv4
Detection ppogg/YOLOv5Lite ?
Detection RangiLyu/NanoDetPlus
KeyPoint PaddleDetection/TinyPose
KeyPoint PaddleDetection/PicoDet + TinyPose
HeadPose omasaht/headpose
Tracking PaddleDetection/PP-Tracking
OCR PaddleOCR/PP-OCRv2
OCR PaddleOCR/PP-OCRv3
Segmentation PaddleSeg/PP-LiteSeg
Segmentation PaddleSeg/PP-HumanSegLite
Segmentation PaddleSeg/HRNet
Segmentation PaddleSeg/PP-HumanSegServer
Segmentation PaddleSeg/Unet
Segmentation PaddleSeg/Deeplabv3
FaceDetection biubug6/RetinaFace
FaceDetection Linzaer/UltraFace
FaceDetection deepcam-cn/YOLOv5Face
FaceDetection insightface/SCRFD
FaceAlign Hsintao/PFLD
FaceAlign Single430/FaceLandmark1000
FaceAlign jhb86253817/PIPNet
FaceRecognition insightface/ArcFace
FaceRecognition insightface/CosFace
FaceRecognition insightface/PartialFC
FaceRecognition insightface/VPL
Matting ZHKKKe/MODNet
Matting PeterL1n/RobustVideoMatting
Matting PaddleSeg/PP-Matting
Matting PaddleSeg/PP-HumanMatting
Matting PaddleSeg/ModNet
Video Super-Resolution PaddleGAN/BasicVSR
Video Super-Resolution PaddleGAN/EDVR
Video Super-Resolution PaddleGAN/PP-MSVSR
Information Extraction PaddleNLP/UIE
NLP PaddleNLP/ERNIE-3.0
Speech PaddleSpeech/PP-TTS --

📲 移动端和端侧部署 🔥🔥🔥🔥

端侧模型支持列表

端侧模型支持列表(点击可收缩)
任务场景 模型 大小(MB) Linux Android Linux Linux Linux Linux Linux 更新中...
--- --- --- ARM CPU ARM CPU 瑞芯微NPU
RK3568/RK3588
瑞芯微NPU
RV1109/RV1126/RK1808
晶晨NPU
A311D/S905D/C308X
恩智浦NPU
i.MX 8M Plus
更新中...|
Classification PaddleClas/ResNet50 98
Classification PaddleClas/PP-LCNet 11.9 -- -- --
Classification PaddleClas/PP-LCNetv2 26.6 -- -- --
Classification PaddleClas/EfficientNet 31.4 -- -- --
Classification PaddleClas/GhostNet 20.8 -- -- --
Classification PaddleClas/MobileNetV1 17 -- -- --
Classification PaddleClas/MobileNetV2 14.2 -- -- --
Classification PaddleClas/MobileNetV3 22 --
Classification PaddleClas/ShuffleNetV2 9.2 -- -- --
Classification PaddleClas/SqueezeNetV1.1 5 -- -- --
Classification PaddleClas/Inceptionv3 95.5 -- -- --
Classification PaddleClas/PP-HGNet 59 -- -- --
Detection PaddleDetection/PicoDet_s 4.9 --
Detection YOLOv5 --
Face Detection deepinsight/SCRFD 2.5 -- -- -- --
Keypoint Detection PaddleDetection/PP-TinyPose 5.5 --
Segmentation PaddleSeg/PP-LiteSeg(STDC1) 32.2 -- -- -- --
Segmentation PaddleSeg/PP-HumanSeg-Lite 0.556 -- -- -- --
Segmentation PaddleSeg/HRNet-w18 38.7 -- -- -- --
Segmentation PaddleSeg/PP-HumanSeg 107.2 -- -- -- --
Segmentation PaddleSeg/Unet 53.7 -- -- -- --
Segmentation PaddleSeg/Deeplabv3 150
OCR PaddleOCR/PP-OCRv2 2.3+4.4 -- -- -- --
OCR PaddleOCR/PP-OCRv3 2.4+10.6 --

🌐 Web和小程序部署

Web和小程序部署支持列表(点击可收缩)
任务场景 模型 web_demo
--- --- Paddle.js
Detection FaceDetection
Detection ScrewDetection
Segmentation PaddleSeg/HumanSeg
Object Recognition GestureRecognition
Object Recognition ItemIdentification
OCR PaddleOCR/PP-OCRv3

社区交流

  • Slack:Join our Slack community and chat with other community members about ideas

  • 微信:扫描二维码,填写问卷加入技术社区,与社区开发者探讨部署的痛点与方案

Acknowledge

本项目中SDK生成和下载使用了EasyEdge中的免费开放能力,在此表示感谢。

License

FastDeploy遵循Apache-2.0开源协议