Headline finding
AlignedServe improves decoding throughput by a maximum of 1.98x and reduces P99 TPOT latency by up to 7.4x compared to state-of-the-art systems (vLLM, DistServe, and DeepSpeed-FastGen) across synthetic and application workloads.
The experimental results in Sections 4.2 and 4.3 support the claimed maximum 1.98x throughput improvement and up to 7.4x P99 TPOT latency reduction.
“The experimental results demonstrate that AlignedServe improves the decoding throughput by a maximum of 1.98× and reduces the latency by up to 7.4× compared to the state-of-the-art systems.” · Show in paper