Headline finding
Implemented on vLLM and evaluated on production-style traces with DeepSeek-V2-Lite, Kairos reduces P95 TTFT by up to 81% and raises SLO attainment by up to 79% over state-of-the-art disaggregated schedulers, at sub-millisecond per-request routing cost.
The evaluation results in Section 6 (Figures 4-6) and Section 4.4/5.3 support the improvements in P95 TTFT, SLO attainment, and sub-millisecond routing overhead, though Section 9 (Conclusion) states 'upto 71%' SLO attainment improvement rather than 79%.
“Implemented on vLLM and evaluated on production-style traces with DeepSeek-V2-Lite, our approach reduces P95 TTFT by upto 81% and raises SLO attainment by upto 79% over state-of-the-art disaggregated schedulers, at sub-millisecond per-request routing cost.” · Show in paper