Headline finding
Prefix-cached LLM agent serving is metastable: a transient trigger such as a few minutes of extra arrivals at a base load the cache-warm server otherwise sustains can push the server into a persistent degraded regime with low hit ratio, halved throughput, and a growing backlog that does not recover after the trigger ends.
The paired experiments, no-cache and session-cap controls, and reproductions across models and engines directly support the claim of metastability.
“We show that this makes prefix-cached agent serving metastable. With an identical base workload, a few minutes of extra arrivals leaves a vLLM server in a degraded regime—a ∼10% hit ratio, about half the throughput, and a backlog that grows for the remaining 70 minutes—while its no-burst twin stays healthy (9 of 12 pairs; none in the reverse direction).” · Show in paper