INFaaS

LLM serving scheduler extended to support multi-tier SLAs

Researchers have extended the Llumnix LLM serving scheduler to support more than two priority tiers, enabling it to better manage heterogeneous service-level objectives (SLOs). The enhanced scheduler was evaluated using…