LLM inference in C/C++

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Protocol Buffers - Google's data interchange format

The C based gRPC (C++, Python, Ruby, Objective-C, PHP, C#)

Carbon Language's main repository: documents, design, implementation, and related tools. (NOTE: Carbon Language is experimental; see README)

Productive, portable, and performant GPU programming in Python.

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

📚 Modern C++ Tutorial: C++11/14/17/20 On the Fly | https://changkun.de/modern-cpp/

ncnn is a high-performance neural network inference framework optimized for the mobile platform

MLX: An array framework for Apple silicon

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning …

Official inference framework for 1-bit LLMs

Tensor library for machine learning

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

Turi Create simplifies the development of custom machine learning models.

The Z3 Theorem Prover

A General-purpose Task-parallel Programming System using Modern C++

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports comp…

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

High-speed Large Language Model Serving for Local Deployment

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.

CUDA Templates for Linear Algebra Subroutines

Transformer related optimization, including BERT, GPT

a language for fast, portable data-parallel computation

oneAPI Threading Building Blocks (oneTBB)

MegEngine 是一个快速、可拓展、易于使用且支持自动求导的深度学习框架

PlaidML is a framework for making deep learning work everywhere.

Extremely simple yet powerful header-only C++ plotting library built on the popular matplotlib

MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.