Load Testing Text Generation Inference on different platforms

We are using k6 to run load testing on different platforms.

Run test

k6 run sharegpt_load.js

Use Environment variables

k6 run sharegpt_load.js -e HOST=https://xxx

Configuration

-e HOST=https://xxx # host url
-e DO_SAMPLE=1 # do sample request

Installation

sudo gpg -k
sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt-get update
sudo apt-get install k6