LLM inference in C/C++

MLX: An array framework for Apple silicon

❤️中国科学技术大学课程资源

High-speed Large Language Model Serving for Local Deployment

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.

Optimized primitives for collective multi-GPU communication

On-device AI across mobile, embedded and edge for PyTorch

Fast, Flexible and Portable Structured Generation

Fast Multimodal LLM on Mobile Devices

Low-bit LLM inference on CPU with lookup table

A GPU benchmark suite for assessing on-chip GPU memory bandwidth

Compiler 2020 cource of USTC labs

Project of the group NooBirds for OSH-2022.

USTC 算法基础 实验 顾乃杰 2021 陈雪 OJ