Releases: casys-kaist/LLMServingSim
Releases · casys-kaist/LLMServingSim
v0.1.0
Performance model update for LLMServingSim
New features
- Support GPU with a performance model
- Auto config generator (network and memory)
- Verbose option for more detailed log
- More metrics (queuing_delay, TTFT, TPOT)
- Refactored code structure for readability
v0.0.0-artifact
IISWC Artifact for "LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale"