LLM inference in C/C++

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

A library for efficient similarity search and clustering of dense vectors.

Tensor library for machine learning

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

High-speed Large Language Model Serving for Local Deployment

Transformer related optimization, including BERT, GPT

基于深度学习高性能中文车牌识别 High Performance Chinese License Plate Recognition Framework.

Fast inference engine for Transformer models

LightSeq: A High Performance Library for Sequence Processing and Generation

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

Track vehicles and persons on rk3588 / rk3399pro.

Useful tensorrt plugin. For pytorch and mmdetection model conversion.

pytorch during training, libtorch during serving via gRPC

rk3588 various solutions for reading camera and video files

测试rknn分类模型与目标检测模型指标

Protocol Buffers - Google's data interchange format

Implementation of popular deep learning networks with TensorRT network definition API