Skip to content

Releases: casys-kaist/LLMServingSim

v0.1.0

03 Jan 06:14
Compare
Choose a tag to compare

Performance model update for LLMServingSim

New features

  • Support GPU with a performance model
  • Auto config generator (network and memory)
  • Verbose option for more detailed log
  • More metrics (queuing_delay, TTFT, TPOT)
  • Refactored code structure for readability

v0.0.0-artifact

03 Jan 06:06
2e63827
Compare
Choose a tag to compare

IISWC Artifact for "LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale"