Llama server

Rest API Server for Llama 2

Prerequisite

Before run this server, you have to download Llama 2 model first.

See https://dev.to/choonho/llama-2-in-apple-silicon-macbook-13-54h

Move ggsm model file to models/7B/ggml-model-q4_0.bin (default MODEL_PATH)

Installation

pip3 install llama-cpp-python langchain
pip3 install fastapi uvicorn

Run Server

python3 server.py