C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)

Falcon LLM ggml framework with CPU and GPU support

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with …

Suno AI's Bark model in C/C++ for fast text-to-speech generation

Web browser version of StarCoder.cpp

Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)

Stable Diffusion and Flux in pure C/C++

Python bindings for ggml

C++ implementation of Qwen-LM

LLaVA server (llama.cpp).

Inference Vision Transformer (ViT) in plain C/C++ with ggml

Run GGML models with Kubernetes.

llama and other large language models on iOS and MacOS offline using GGML library.

High-speed Large Language Model Serving for Local Deployment

Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)

GGML implementation of BERT model with Python bindings and quantization.