init commit

.gitattributes

@@ -0,0 +1,9 @@
+*.tensor filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.so filter=lfs diff=lfs merge=lfs -text
+*.cache filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.so.* filter=lfs diff=lfs merge=lfs -text
+*.cache.* filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+assets/cuda-bevfusion.gif filter=lfs diff=lfs merge=lfs -text

.gitignore

@@ -0,0 +1 @@
+__pycache__

CLA.md

@@ -0,0 +1,58 @@
+## Individual Contributor License Agreement (CLA)
+
+**Thank you for submitting your contributions to this project.**
+
+By signing this CLA, you agree that the following terms apply to all of your past, present and future contributions
+to the project.
+
+### License.
+
+You hereby represent that all present, past and future contributions are governed by the
+[MIT License](https://opensource.org/licenses/MIT)
+copyright statement.
+
+This entails that to the extent possible under law, you transfer all copyright and related or neighboring rights
+of the code or documents you contribute to the project itself or its maintainers.
+Furthermore you also represent that you have the authority to perform the above waiver
+with respect to the entirety of you contributions.
+
+### Moral Rights.
+
+To the fullest extent permitted under applicable law, you hereby waive, and agree not to
+assert, all of your “moral rights” in or relating to your contributions for the benefit of the project.
+
+### Third Party Content.
+
+If your Contribution includes or is based on any source code, object code, bug fixes, configuration changes, tools,
+specifications, documentation, data, materials, feedback, information or other works of authorship that were not
+authored by you (“Third Party Content”) or if you are aware of any third party intellectual property or proprietary
+rights associated with your Contribution (“Third Party Rights”),
+then you agree to include with the submission of your Contribution full details respecting such Third Party
+Content and Third Party Rights, including, without limitation, identification of which aspects of your
+Contribution contain Third Party Content or are associated with Third Party Rights, the owner/author of the
+Third Party Content and Third Party Rights, where you obtained the Third Party Content, and any applicable
+third party license terms or restrictions respecting the Third Party Content and Third Party Rights. For greater
+certainty, the foregoing obligations respecting the identification of Third Party Content and Third Party Rights
+do not apply to any portion of a Project that is incorporated into your Contribution to that same Project.
+
+### Representations.
+
+You represent that, other than the Third Party Content and Third Party Rights identified by
+you in accordance with this Agreement, you are the sole author of your Contributions and are legally entitled
+to grant the foregoing licenses and waivers in respect of your Contributions. If your Contributions were
+created in the course of your employment with your past or present employer(s), you represent that such
+employer(s) has authorized you to make your Contributions on behalf of such employer(s) or such employer
+(s) has waived all of their right, title or interest in or to your Contributions.
+
+### Disclaimer.
+
+To the fullest extent permitted under applicable law, your Contributions are provided on an "as is"
+basis, without any warranties or conditions, express or implied, including, without limitation, any implied
+warranties or conditions of non-infringement, merchantability or fitness for a particular purpose. You are not
+required to provide support for your Contributions, except to the extent you desire to provide support.
+
+### No Obligation.
+
+You acknowledge that the maintainers of this project are under no obligation to use or incorporate your contributions
+into the project. The decision to use or incorporate your contributions into the project will be made at the
+sole discretion of the maintainers or their authorized delegates.

LICENSE.md

@@ -0,0 +1,24 @@
+# MIT License
+
+<pre>
+SPDX-FileCopyrightText: Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+SPDX-License-Identifier: MIT
+
+Permission is hereby granted, free of charge, to any person obtaining a
+copy of this software and associated documentation files (the "Software"),
+to deal in the Software without restriction, including without limitation
+the rights to use, copy, modify, merge, publish, distribute, sublicense,
+and/or sell copies of the Software, and to permit persons to whom the
+Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+DEALINGS IN THE SOFTWARE.
+</pre>

README.md

@@ -0,0 +1,101 @@
+<h1 style="text-align: center">Lidar AI Solution</h1>
+This is a highly optimized solution for self-driving 3D-lidar repository.
+It does a great job of speeding up sparse convolution/CenterPoint/BEVFusion/OSD/Conversion.
+
+![title](assets/title.png)
+
+## Pipeline overview
+![pipeline](assets/pipeline.png)
+
+## GetStart
+```
+$ sudo apt-get install git-lfs
+$ git clone --recursive https://github.com/NVIDIA-AI-IOT/Lidar_AI_Solution
+$ cd Lidar_AI_Solution
+```
+- For each specific task please refer to the readme in the sub-folder.
+
+## 3D Sparse Convolution
+A tiny inference engine for [3d sparse convolutional networks](https://github.com/tianweiy/CenterPoint/blob/master/det3d/models/backbones/scn.py) using int8/fp16.
+- **Tiny Engine:** Tiny Lidar-Backbone inference engine independent of TensorRT.
+- **Flexible:** Build execution graph from ONNX.
+- **Easy To Use:** Simple interface and onnx export solution.
+- **High Fidelity:** Low accuracy drop on nuScenes validation.
+- **Low Memory:** 422MB@SCN FP16, 426MB@SCN INT8.
+- **Compact:** Based on the CUDA kernels and independent of cutlass.
+
+## CUDA BEVFusion
+CUDA & TensorRT solution for [BEVFusion](https://arxiv.org/abs/2205.13542) inference, including:
+- **Camera Encoder**: Swin-Transformer and finetuned BEV pooling with TensorRT and onnx export solution.
+- **Lidar Encoder**: Tiny Lidar-Backbone inference independent of TensorRT and onnx export solution.
+- **Feature Fusion**: Camera & Lidar feature fuser with TensorRT and onnx export solution.
+- **Pre/Postprocess**: Interval precomputing, lidar voxelization, feature decoder with CUDA kernels.
+- **Easy To Use**: Preparation, inference, evaluation all in one to reproduce torch Impl accuracy.
+- **PTQ**: Quantization solutions for [mmdet3d/spconv](https://github.com/mit-han-lab/bevfusion/tree/main/mmdet3d/ops/spconv), Easy to understand.
+
+## CUDA CenterPoint
+CUDA & TensorRT solution for [CenterPoint](https://arxiv.org/abs/2006.11275) inference, including:
+- **Preprocess**: Voxelization with CUDA kernel
+- **Encoder**: 3D backbone with NV spconv-scn and onnx export solution.
+- **Neck & Header**: RPN & CenterHead with TensorRT and onnx export solution.
+- **Postprocess**: Decode & NMS with CUDA kernel
+- **Easy To Use**: Preparation, inference, evaluation all in one to reproduce torch Impl accuracy.
+- **QAT**: Quantization solutions for [traveller59/spconv](https://github.com/mit-han-lab/bevfusion/tree/main/mmdet3d/ops/spconv), Easy to understand.
+
+## CUDA PointPillars
+CUDA & TensorRT solution for [pointpillars](https://arxiv.org/abs/1812.05784) inference, including:
+- **Preprocess**: Voxelization & Feature Extending with CUDA kernel
+- **Detector**: 2.5D backbone with TensorRT and onnx export solution.
+- **Postprocess**: Parse bounding box, class type and direction
+- **Easy To Use**: Preparation, inference, evaluation all in one to reproduce torch Impl accuracy.
+
+
+## cuOSD(CUDA On-Screen Display Library)
+Draw all elements using a single CUDA kernel.
+- **Line:** Plotting lines by interpolation(Nearest or Linear).
+- **RotateBox:** Supports drawn with different border colors and fill colors.
+- **Circle:** Supports drawn with different border colors and fill colors.
+- **Rectangle:** Supports drawn with different border colors and fill colors.
+- **Text:** Supports [stb_truetype](https://github.com/nothings/stb/blob/master/stb_truetype.h) and [pango-cairo](https://pango.gnome.org/) backends, allowing fonts to be read via TTF or using font-family.
+- **Arrow:** Combination of arrows by 3 lines.
+- **Point:** Plotting points by interpolation(Nearest or Linear).
+- **Clock:** Time plotting based on text support
+
+## cuPCL(CUDA Point Cloud Library)
+Provide several GPU accelerated Point Cloud operations with high accuracy and high perfomrance at the same time: cuICP, cuFilter, cuSegmentation, cuOctree, cuCluster, cuNDT, Voxelization(incoming).
+- **cuICP:** CUDA accelerated iterative corresponding point vertex cloud(point-to-point) registration implementation.
+- **cuFilter:** Support CUDA accelerated features: PassThrough and VoxelGrid.
+- **cuSegmentation:** Support CUDA accelerated features: RandomSampleConsensus with a plane model.
+- **cuOctree:** Support CUDA accelerated features: Approximate Nearest Search and Radius Search.
+- **cuCluster:** Support CUDA accelerated features: Cluster based on the distance among points.
+- **cuNDT:** CUDA accelerated 3D Normal Distribution Transform registration implementation for point cloud data.
+
+## YUVToRGB(CUDA Conversion)
+YUV to RGB conversion. Combine Resize/Padding/Conversion/Normalization into a single kernel function.
+- **Most of the time, it can be bit-aligned with OpenCV.**
+    - It will give an exact result when the scaling factor is a rational number.
+    - Better performance is usually achieved when the stride can divide by 4.
+- Supported Input Format:
+    1. **NV12BlockLinear**
+    2. **NV12PitchLinear**
+    3. **YUV422Packed_YUYV**
+- Supported Interpolation methods:
+    1. **Nearest**
+    2. **Bilinear**
+- Supported Output Data Type:
+    1. **Uint8**
+    2. **Float32**
+    3. **Float16**
+- Supported Output Layout:
+    1. **CHW_RGB/BGR**
+    2. **HWC_RGB/BGR**
+    3. **CHW16/32/4/RGB/BGR for DLA input**
+
+## Thanks
+This project makes use of a number of awesome open source libraries, including:
+
+- [stb_image](https://github.com/nothings/stb) for PNG and JPEG support
+- [pybind11](https://github.com/pybind/pybind11) for seamless C++ / Python interop
+- and others! See the dependencies folder.
+
+Many thanks to the authors of these brilliant projects!