Headline finding
Across several memory-sensitive MoE serving scenarios, PagedWeight improves the quality-memory tradeoff over existing quantization baselines, achieving FP16-equivalent accuracy with up to 72.0% GPU memory savings and 1.94× throughput improvement, and improving quality over quantization methods by up to 39.3% at a similar memory budget with at most 4.1% throughput loss.
The reported figures match the experimental tables: (35.25 - 9.86)/35.25 = 72.0% memory savings, 130.1/67.1 = 1.94× throughput, (17.0 - 12.2)/12.2 = 39.3% quality gain, and (401.9 - 385.4)/401.9 = 4.1% max throughput loss.
“Across several memory-sensitive MoE serving scenarios, PagedWeight improves the quality-memory tradeoff over several existing quantization baselines. PagedWeight achieves FP16-equivalent accuracy with up to 72.0% GPU memory savings and 1.94× throughput improvement, and improves quality over quantization methods by up to 39.3% at a similar memory budget with at most 4.1% throughput loss.” · Show in paper