Headline finding
ARCQuant boosts NVFP4 quantization performance for LLMs via Augmented Residual Channels, maintaining a strictly unified NVFP4 format to achieve state-of-the-art accuracy comparable to full-precision baselines in perplexity and downstream tasks, while achieving up to 3x speedup over FP16 on RTX 5090 and RTX PRO 6000 GPUs.
The experimental results in Table 1, Figure 6, and Table 10 directly support the accuracy and speedup claims on LLaMA and Qwen models across RTX 5090 and RTX PRO 6000 GPUs.
“To address these challenges, we propose ARCQuant, a framework that boosts NVFP4 performance via Augmented Residual Channels. Distinct from methods that compromise block isolation or hardware uniformity, ARCQuant maintains a strictly unified NVFP4 format by augmenting the activation matrix with quantized residual channels. ... Extensive experiments on LLaMA and Qwen models demonstrate that ARCQuant achieves state-of-the-art accuracy, comparable to full-precision baselines in perplexity and downstream tasks. Furthermore, deployment on RTX 5090 and RTX PRO 6000 GPUs confirms practical benefits, achieving up to 3× speedup over FP16.” · Show in paper