Add model YOLOR Support (PaddlePaddle#31)

* first commit for yolov7 * pybind for yolov7 * CPP README.md * CPP README.md * modified yolov7.cc * README.md * python file modify * delete license in fastdeploy/ * repush the conflict part * README.md modified * README.md modified * file path modified * file path modified * file path modified * file path modified * file path modified * README modified * README modified * move some helpers to private * add examples for yolov7 * api.md modified * api.md modified * api.md modified * YOLOv7 * yolov7 release link * yolov7 release link * yolov7 release link * copyright * change some helpers to private * change variables to const and fix documents. * gitignore * Transfer some funtions to private member of class * Transfer some funtions to private member of class * Merge from develop (PaddlePaddle#9) * Fix compile problem in different python version (PaddlePaddle#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <[email protected]> * Add PaddleDetetion/PPYOLOE model support (PaddlePaddle#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (PaddlePaddle#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (PaddlePaddle#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> * first commit for yolor * for merge * Develop (PaddlePaddle#11) (PaddlePaddle#12) * Fix compile problem in different python version (PaddlePaddle#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <[email protected]> * Add PaddleDetetion/PPYOLOE model support (PaddlePaddle#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (PaddlePaddle#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (PaddlePaddle#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> * Develop (PaddlePaddle#13) * Fix compile problem in different python version (PaddlePaddle#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <[email protected]> * Add PaddleDetetion/PPYOLOE model support (PaddlePaddle#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (PaddlePaddle#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (PaddlePaddle#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * documents * Develop (PaddlePaddle#14) * Fix compile problem in different python version (PaddlePaddle#26) * fix some usage problem in linux * Fix compile problem Co-authored-by: root <[email protected]> * Add PaddleDetetion/PPYOLOE model support (PaddlePaddle#22) * add ppdet/ppyoloe * Add demo code and documents * add convert processor to vision (PaddlePaddle#27) * update .gitignore * Added checking for cmake include dir * fixed missing trt_backend option bug when init from trt * remove un-need data layout and add pre-check for dtype * changed RGB2BRG to BGR2RGB in ppcls model * add model_zoo yolov6 c++/python demo * fixed CMakeLists.txt typos * update yolov6 cpp/README.md * add yolox c++/pybind and model_zoo demo * move some helpers to private * fixed CMakeLists.txt typos * add normalize with alpha and beta * add version notes for yolov5/yolov6/yolox * add copyright to yolov5.cc * revert normalize * fixed some bugs in yolox * fixed examples/CMakeLists.txt to avoid conflicts * add convert processor to vision * format examples/CMakeLists summary * Fix bug while the inference result is empty with YOLOv5 (PaddlePaddle#29) * Add multi-label function for yolov5 * Update README.md Update doc * Update fastdeploy_runtime.cc fix variable option.trt_max_shape wrong name * Update runtime_option.md Update resnet model dynamic shape setting name from images to x * Fix bug when inference result boxes are empty * Delete detection.py Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> Co-authored-by: Jason <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: DefTruth <[email protected]> Co-authored-by: huangjianhui <[email protected]> Co-authored-by: Jason <[email protected]>
marsbzp · Jul 21, 2022 · 8b0a0c6 · 8b0a0c6
1 parent 39aca67
commit 8b0a0c6
Show file tree

Hide file tree

Showing 21 changed files with 820 additions and 32 deletions.
diff --git a/.gitignore b/.gitignore
@@ -11,4 +11,4 @@ fastdeploy.egg-info
 .setuptools-cmake-build
 fastdeploy/version.py
 fastdeploy/LICENSE*
-fastdeploy/ThirdPartyNotices*
+fastdeploy/ThirdPartyNotices*
diff --git a/examples/vision/wongkinyiu_yolor.cc b/examples/vision/wongkinyiu_yolor.cc
@@ -0,0 +1,52 @@
+// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "fastdeploy/vision.h"
+
+int main() {
+  namespace vis = fastdeploy::vision;
+
+  std::string model_file = "../resources/models/yolor.onnx";
+  std::string img_path = "../resources/images/horses.jpg";
+  std::string vis_path = "../resources/outputs/wongkinyiu_yolor_vis_result.jpg";
+
+  auto model = vis::wongkinyiu::YOLOR(model_file);
+  if (!model.Initialized()) {
+    std::cerr << "Init Failed! Model: " << model_file << std::endl;
+    return -1;
+  } else {
+    std::cout << "Init Done! Model:" << model_file << std::endl;
+  }
+  model.EnableDebug();
+
+  cv::Mat im = cv::imread(img_path);
+  cv::Mat vis_im = im.clone();
+
+  vis::DetectionResult res;
+  if (!model.Predict(&im, &res)) {
+    std::cerr << "Prediction Failed." << std::endl;
+    return -1;
+  } else {
+    std::cout << "Prediction Done!" << std::endl;
+  }
+
+  // 输出预测框结果
+  std::cout << res.Str() << std::endl;
+
+  // 可视化预测结果
+  vis::Visualize::VisDetection(&vis_im, res);
+  cv::imwrite(vis_path, vis_im);
+  std::cout << "Detect Done! Saved: " << vis_path << std::endl;
+  return 0;
+}
diff --git a/fastdeploy/utils/utils.h b/fastdeploy/utils/utils.h
@@ -26,10 +26,10 @@
 #define FASTDEPLOY_DECL __declspec(dllexport)
 #else
 #define FASTDEPLOY_DECL __declspec(dllimport)
-#endif // FASTDEPLOY_LIB
+#endif  // FASTDEPLOY_LIB
 #else
 #define FASTDEPLOY_DECL __attribute__((visibility("default")))
-#endif // _WIN32
+#endif  // _WIN32
 
 namespace fastdeploy {
 
@@ -42,7 +42,8 @@ class FASTDEPLOY_DECL FDLogger {
   }
   explicit FDLogger(bool verbose, const std::string& prefix = "[FastDeploy]");
 
-  template <typename T> FDLogger& operator<<(const T& val) {
+  template <typename T>
+  FDLogger& operator<<(const T& val) {
     if (!verbose_) {
       return *this;
     }
@@ -72,10 +73,14 @@ class FASTDEPLOY_DECL FDLogger {
   FDLogger(true, "[ERROR]")                                                    \
       << __REL_FILE__ << "(" << __LINE__ << ")::" << __FUNCTION__ << "\t"
 
-#define FDASSERT(condition, message)                                           \
-  if (!(condition)) {                                                          \
-    FDERROR << message << std::endl;                                           \
-    std::abort();                                                              \
+#define FDERROR                                                \
+  FDLogger(true, "[ERROR]") << __REL_FILE__ << "(" << __LINE__ \
+                            << ")::" << __FUNCTION__ << "\t"
+
+#define FDASSERT(condition, message) \
+  if (!(condition)) {                \
+    FDERROR << message << std::endl; \
+    std::abort();                    \
   }
 
-} // namespace fastdeploy
+}  // namespace fastdeploy
diff --git a/fastdeploy/vision.h b/fastdeploy/vision.h
@@ -15,12 +15,13 @@
 
 #include "fastdeploy/core/config.h"
 #ifdef ENABLE_VISION
+#include "fastdeploy/vision/megvii/yolox.h"
+#include "fastdeploy/vision/meituan/yolov6.h"
 #include "fastdeploy/vision/ppcls/model.h"
 #include "fastdeploy/vision/ppdet/ppyoloe.h"
 #include "fastdeploy/vision/ultralytics/yolov5.h"
+#include "fastdeploy/vision/wongkinyiu/yolor.h"
 #include "fastdeploy/vision/wongkinyiu/yolov7.h"
-#include "fastdeploy/vision/meituan/yolov6.h"
-#include "fastdeploy/vision/megvii/yolox.h"
 #endif
 
 #include "fastdeploy/vision/visualize/visualize.h"
diff --git a/fastdeploy/vision/ppcls/model.h b/fastdeploy/vision/ppcls/model.h
@@ -46,6 +46,6 @@ class FASTDEPLOY_DECL Model : public FastDeployModel {
   std::vector<std::shared_ptr<Processor>> processors_;
   std::string config_file_;
 };
-} // namespace ppcls
-} // namespace vision
-} // namespace fastdeploy
+}  // namespace ppcls
+}  // namespace vision
+}  // namespace fastdeploy
diff --git a/fastdeploy/vision/ppcls/ppcls_pybind.cc b/fastdeploy/vision/ppcls/ppcls_pybind.cc
@@ -27,4 +27,4 @@ void BindPPCls(pybind11::module& m) {
              return res;
            });
 }
-} // namespace fastdeploy
+}  // namespace fastdeploy
diff --git a/fastdeploy/vision/visualize/detection.cc b/fastdeploy/vision/visualize/detection.cc
@@ -56,6 +56,6 @@ void Visualize::VisDetection(cv::Mat* im, const DetectionResult& result,
   }
 }
 
-} // namespace vision
-} // namespace fastdeploy
+}  // namespace vision
+}  // namespace fastdeploy
 #endif
diff --git a/fastdeploy/vision/wongkinyiu/__init__.py b/fastdeploy/vision/wongkinyiu/__init__.py
@@ -114,3 +114,101 @@ def max_wh(self, value):
         assert isinstance(
             value, float), "The value to set `max_wh` must be type of float."
         self._model.max_wh = value
+
+
+class YOLOR(FastDeployModel):
+    def __init__(self,
+                 model_file,
+                 params_file="",
+                 runtime_option=None,
+                 model_format=Frontend.ONNX):
+        # 调用基函数进行backend_option的初始化
+        # 初始化后的option保存在self._runtime_option
+        super(YOLOR, self).__init__(runtime_option)
+
+        self._model = C.vision.wongkinyiu.YOLOR(
+            model_file, params_file, self._runtime_option, model_format)
+        # 通过self.initialized判断整个模型的初始化是否成功
+        assert self.initialized, "YOLOR initialize failed."
+
+    def predict(self, input_image, conf_threshold=0.25, nms_iou_threshold=0.5):
+        return self._model.predict(input_image, conf_threshold,
+                                   nms_iou_threshold)
+
+    # 一些跟YOLOR模型有关的属性封装
+    # 多数是预处理相关，可通过修改如model.size = [1280, 1280]改变预处理时resize的大小（前提是模型支持）
+    @property
+    def size(self):
+        return self._model.size
+
+    @property
+    def padding_value(self):
+        return self._model.padding_value
+
+    @property
+    def is_no_pad(self):
+        return self._model.is_no_pad
+
+    @property
+    def is_mini_pad(self):
+        return self._model.is_mini_pad
+
+    @property
+    def is_scale_up(self):
+        return self._model.is_scale_up
+
+    @property
+    def stride(self):
+        return self._model.stride
+
+    @property
+    def max_wh(self):
+        return self._model.max_wh
+
+    @size.setter
+    def size(self, wh):
+        assert isinstance(wh, [list, tuple]),\
+            "The value to set `size` must be type of tuple or list."
+        assert len(wh) == 2,\
+            "The value to set `size` must contatins 2 elements means [width, height], but now it contains {} elements.".format(
+            len(wh))
+        self._model.size = wh
+
+    @padding_value.setter
+    def padding_value(self, value):
+        assert isinstance(
+            value,
+            list), "The value to set `padding_value` must be type of list."
+        self._model.padding_value = value
+
+    @is_no_pad.setter
+    def is_no_pad(self, value):
+        assert isinstance(
+            value, bool), "The value to set `is_no_pad` must be type of bool."
+        self._model.is_no_pad = value
+
+    @is_mini_pad.setter
+    def is_mini_pad(self, value):
+        assert isinstance(
+            value,
+            bool), "The value to set `is_mini_pad` must be type of bool."
+        self._model.is_mini_pad = value
+
+    @is_scale_up.setter
+    def is_scale_up(self, value):
+        assert isinstance(
+            value,
+            bool), "The value to set `is_scale_up` must be type of bool."
+        self._model.is_scale_up = value
+
+    @stride.setter
+    def stride(self, value):
+        assert isinstance(
+            value, int), "The value to set `stride` must be type of int."
+        self._model.stride = value
+
+    @max_wh.setter
+    def max_wh(self, value):
+        assert isinstance(
+            value, float), "The value to set `max_wh` must be type of float."
+        self._model.max_wh = value
diff --git a/fastdeploy/vision/wongkinyiu/wongkinyiu_pybind.cc b/fastdeploy/vision/wongkinyiu/wongkinyiu_pybind.cc
@@ -17,7 +17,7 @@
 namespace fastdeploy {
 void BindWongkinyiu(pybind11::module& m) {
   auto wongkinyiu_module =
-      m.def_submodule("wongkinyiu", "https://github.com/WongKinYiu/yolov7");
+      m.def_submodule("wongkinyiu", "https://github.com/WongKinYiu");
   pybind11::class_<vision::wongkinyiu::YOLOv7, FastDeployModel>(
       wongkinyiu_module, "YOLOv7")
       .def(pybind11::init<std::string, std::string, RuntimeOption, Frontend>())
@@ -37,5 +37,24 @@ void BindWongkinyiu(pybind11::module& m) {
       .def_readwrite("is_scale_up", &vision::wongkinyiu::YOLOv7::is_scale_up)
       .def_readwrite("stride", &vision::wongkinyiu::YOLOv7::stride)
       .def_readwrite("max_wh", &vision::wongkinyiu::YOLOv7::max_wh);
+
+  pybind11::class_<vision::wongkinyiu::YOLOR, FastDeployModel>(
+      wongkinyiu_module, "YOLOR")
+      .def(pybind11::init<std::string, std::string, RuntimeOption, Frontend>())
+      .def("predict",
+           [](vision::wongkinyiu::YOLOR& self, pybind11::array& data,
+              float conf_threshold, float nms_iou_threshold) {
+             auto mat = PyArrayToCvMat(data);
+             vision::DetectionResult res;
+             self.Predict(&mat, &res, conf_threshold, nms_iou_threshold);
+             return res;
+           })
+      .def_readwrite("size", &vision::wongkinyiu::YOLOR::size)
+      .def_readwrite("padding_value", &vision::wongkinyiu::YOLOR::padding_value)
+      .def_readwrite("is_mini_pad", &vision::wongkinyiu::YOLOR::is_mini_pad)
+      .def_readwrite("is_no_pad", &vision::wongkinyiu::YOLOR::is_no_pad)
+      .def_readwrite("is_scale_up", &vision::wongkinyiu::YOLOR::is_scale_up)
+      .def_readwrite("stride", &vision::wongkinyiu::YOLOR::stride)
+      .def_readwrite("max_wh", &vision::wongkinyiu::YOLOR::max_wh);
 }
 }  // namespace fastdeploy