Headline finding
MOMENTKV, which maintains compact moment statistics over the evicted token set to jointly guide eviction and provide a closed-form first-order approximation of the evicted attention output, outperforms all tested KV cache eviction baselines on LongBench and RULER with LLaMA-3.1-8B-Instruct and Qwen3-4B-Instruct at every cache budget, with the largest gains under aggressive compression.
The experimental results in Tables 1 and 2 directly support the claim across all tested models, benchmarks, and cache budgets.
“On LongBench and RULER with LLaMA-3.1-8B-Instruct and Qwen3-4B-Instruct, MOMENTKV outperforms all baselines at every cache budget, with the largest gains under aggressive compression.” · Show in paper