Search publications
QQQ: Quality Quattuor-Bit Quantization for Large Language Models
PaperCodeSource excerpt
“The throughput comparison outlined in Figure 6 indicates that QQQ outperforms the alternative methods across a range of batch sizes in most cases. Specifically, QQQ attains a substantial speed increase, reaching up to 2.24×, 2.10×, 1.59×, and 1.25× faster than FP16, SmoothQuant, AWQ, and Marlin on the LLaMA-2-13B model, respectively.” · Show in paper
Verified
Figure 6(b) and Section 4.3 support the reported speedup numbers on LLaMA-2-13B.
ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs
PaperCodeSource excerpt
“Although tailored for the fine-grained NVFP4 format, ARCQuant demonstrates robust extensibility to INT4 and MXFP4 formats. On Llama 3.1-8B, our method consistently outperforms RTN baselines across all metrics as detailed in Appendix B. Notably, ARCQuant reduces INT4 perplexity by 0.89 and improves zero-shot accuracy by 2.12%.” · Show in paper
Verified
Table 7 confirms that ARCQuant improves INT4 average accuracy from 65.78 to 67.90 (+2.12%) and reduces PPL from 8.84 to 7.95 (-0.89), and also improves MXFP4.
End-to-End Context Compression at Scale· Method
PaperCodeSource excerpt
“Stage 0: Adapter warmup. We freeze both the encoder and decoder and update only the adapter. • Stage 1: Encoder training. We unfreeze the encoder while keeping the decoder frozen. • Stage 2: End-to-end continual pre-training. We unfreeze the decoder with a small learning rate. • Stage 3: Supervised fine-tuning. We train on the SFT mixture and increase the decoder learning rate.” · Show in paper
QQQ: Quality Quattuor-Bit Quantization for Large Language Models
PaperCodeSource excerpt
“The per-group QQQ demonstrates competitive performance with with W8A8 SmoothQuant, W4A16 GPTQ, and W4A16 AWQ across various models. For example, per-group QQQ only increases perplexity by up to 0.13 on LLaMA-2-13B compared with them. When benchmarked against W4A4 Atom, both the per-channel and per-group QQQ consistently outperform Atom on all tested models.” · Show in paper
Verified
Table 1 and Table 2 directly substantiate the perplexity and zero-shot accuracy comparisons against SmoothQuant, GPTQ, AWQ, and Atom.
SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference
PaperCodeSource excerpt
“Storage: spending the same CPU memory on exact KV for a subset of spans costs 2.9 to 3.5 points, because low-rank factors cover every span, and a fixed rank of 32 matches the learned rank in accuracy but needs 1.6× the CPU memory.” · Show in paper
Verified
Table 3 directly supports both the 2.9–3.5 point drop for exact KV at matched CPU bytes and the ~1.6x CPU memory increase for fixed rank 32.
MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference
PaperSource excerpt
“Figure 2(d) confirms the amplification empirically: a network-wide average mass below 0.08 produces over 20% relative error on average, with peaks above 60% in the shallowest layers.” · Show in paper
Verified
Figure 2(a)-(d) directly reports these exact averages and peak values.
gpt-oss-120b & gpt-oss-20b Model Card
PaperCodeSource excerpt
“gpt-oss-120b surpasses OpenAI o3-mini and approaches OpenAI o4-mini accuracy. The smaller gpt-oss-20b model is also surprisingly competitive, despite being 6 times smaller than gpt-oss-120b.” · Show in paper
Verified
The reported benchmark scores in Figures 1 and 2 and Table 3 directly support that gpt-oss-120b exceeds o3-mini and approaches (or slightly exceeds on HLE and Tau-Bench Retail) o4-mini, while gpt-oss-20b achieves competitive scores.
Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving
PaperCodeSource excerpt
“SnapKV achieves the best throughput, on par with All-KV on Llama and even surpassing All-KV on Mistral.” · Show in paper
Verified
Figure 3 directly supports that SnapKV-0.75 achieves the highest average throughput among all evaluated methods on both models.
gpt-oss-120b & gpt-oss-20b Model Card· Method
PaperCodeSource excerpt
“Each MoE block consists of a fixed number of experts (128 for gpt-oss-120b and 32 for gpt-oss-20b), as well as a standard linear router projection which maps residual activations to scores for each expert. For both models, we select the top-4 experts for each token given by the router, and weight the output of each expert by the softmax of the router projection over only the selected experts.” · Show in paper
Learning New Facts with QLoRA: An Acquisition-Retention Frontier
PaperCodeSource excerpt
“FFT and QLoRA r = 8 retain OOD performance well but do not reach the highest paraphrase accuracy. Higher-rank QLoRA configurations achieve stronger paraphrase performance only with larger OOD losses. This suggests that the apparent robustness of low-rank adaptation to forgetting is actually partly due to limited plasticity.” · Show in paper
Verified
Table 2 directly supports the claim that low-rank QLoRA and FFT do not reach the highest acquisition targets, while higher ranks do so at the cost of OOD retention.
Bursts That Outlast Themselves: Persistent Degradation in Prefix-Cached Agent Serving
PaperSource excerpt
“Table 2 lists all completed pairs: in 9 of 12 only the burst run degraded persistently, in 0 only the no-burst run did, 2 degraded in both arms and 1 in neither (two-sided sign test over discordant pairs, p = 0.00391).” · Show in paper
Verified
The paired design, statistical test, and control experiments cleanly establish history-dependent persistent degradation on vLLM.
End-to-End Context Compression at Scale
PaperCodeSource excerpt
“Training at scale confirms the choices from the small-scale sweep: both the continual pre-training losses (Figures 9 to 11) and the downstream ablations (Appendix F.6) again favor an encoder window W = 1024 and causal masking. At scale, the attention-based adapter is competitive with the MLP adapter; we keep the MLP adapter, which adds less computation. The choice of pooling operator depends on the compression ratio: mean pooling has a small edge over concatenation at 16x compression, while concatenation slightly outperforms mean pooling at 4x.” · Show in paper
Verified
Both the from-scratch pre-training sweeps and the full-scale continual pre-training and downstream ablations support these architectural conclusions.
KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache· Method
PaperCodeSource excerpt
“our key idea to solve this problem is to group key cache every G tokens and quantize them separately. Because the number of tokens in X_K can be arbitrary, we split X_K into two parts, namely, the grouped part = X_K[: l - r] and residual part = X_K[l - r :], where l is the number of tokens inside the current key cache X_K, r is the number of residual tokens, where l - r can be divisible by G.” · Show in paper
SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?· Method
PaperCodeSource excerpt
“We use the SWE-Agent [16] scaffold. ... For open-source LLMs, we use vllm to host each model. Models are hosted on a single node, with 8 H100 Nvidia GPUs. ... Models have a maximum of 50 turns. We use the same prompt for all models, which is the default prompt from [16].” · Show in paper
RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
PaperCodeSource excerpt
“Figure 8 demonstrates the results on Llama3.1-8B-Ins across multiple different benchmarks. In all cases, HSA consistently outperforms Quest and SparQ, especially at low token budgets.” · Show in paper
Verified
Figure 8 and Tables 2, 6-9 confirm that standalone HSA outperforms Quest and SparQ across the evaluated benchmarks on Llama3.1-8B-Ins.
BridgeKV: Large-Model Prefill and Small-Model Decoding for LLM Inference via KV Cache Translation· Method
PaperSource excerpt
“Gψ,j(z) = Pj z̄ + bj + αj Dj σ(Uj z̄), z̄ = LN(z)” · Show in paper
SoScale: Second-Order Scale Search for FP4 Quantization· Method
PaperSource excerpt
“For methods that require calibration, we sample 128 sequences of 2,048 tokens from the first WikiText shard (Merity et al. 2016), following a widely used PTQ setup (Frantar et al. 2023).” · Show in paper
Learning New Facts with QLoRA: An Acquisition-Retention Frontier· Method
PaperCodeSource excerpt
“We run a separate reasoning experiment on a 94k-example subset of OpenR1-Math-220k (Hugging Face, 2025) to test whether the OSM trend also appears in a larger skill-adaptation regime. We evaluate Pass@1 on MATH-500 (Hendrycks et al., 2021), AIME’24 and AIME’25 (Mathematical Association of America, 2024), AMC’23 (American Mathematics Competitions, 2023), Minerva Math (Lewkowycz et al., 2022), and OlympiadBench (He et al., 2024). OOD degradation uses the same five-benchmark average as the main experiment, relative to the base Qwen3-4B” · Show in paper
AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning
PaperCodeSource excerpt
“Under that control the ranking has no measurable advantage (Appendix N lists every setting). Across five models and three budgets, top-K beats the median matched draw in 11 of 15 settings, and 50 of the 150 draws beat top-K where chance predicts 75. A sign test does not separate that from chance (p=.12), and the median gap is 0.01 perplexity.” · Show in paper
Verified
The parameter-matched control in Table 19 and downstream comparisons in Section 7.2 directly support this negative result.
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
PaperSource excerpt
“As shown in Table 8, compared to an average gold patch, model generated patch files that apply correctly are less than half the total length (74.5 versus 30.1 lines) of gold edit patch files, and rarely edit more than a single file.” · Show in paper
Verified
Table 8 confirms that all evaluated models' applied patches average at most 30.1 total lines (less than half of All Gold's 74.5 lines) and 1.0-1.1 files edited.