Search publications
Learning New Facts with QLoRA: An Acquisition-Retention Frontier
PaperCodeSource excerpt
“The unquantized Qwen3-1.7B control shows the same qualitative monotonic trade-off: paraphrase EM rises from 76% at r = 8 to 79% at r = 16 and 86% at r = 32, while average OOD performance falls from 57.0% to 52.0% and 40.2%, respectively. This suggests that quantization is not required for the qualitative rank trend, although this reduced control changes model scale.” · Show in paper
Verified
The reported numbers on Qwen3-1.7B demonstrate the same monotonic trade-off without quantization, and the authors appropriately qualify the difference in model scale.
FP4Quant: Efficient 4-Bit Communication Compression for Model-Parallel LLM Training
PaperSource excerpt
“Overall, FP4Quant enables efficient low-bit communication with substantial acceleration under TP while incurring negligible runtime overhead under PP, demonstrating its effectiveness across diverse parallel configurations.” · Show in paper
Verified
Table 2 supports both the TP speedup and the negligible overhead under PP.
PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization· Method
PaperSource excerpt
“For each action, the planner estimates accuracy impact by combining three factors: offline global sensitivity, online routing statistics, and runtime prompt residual.” · Show in paper
Harness-Bench: Evaluating Agent Harnesses across Models and Realistic Workflows
PaperCodeDataSource excerpt
“Higher mean performance is associated with smaller variation (Figure 3(a)): Spearman’s ρ = −0.76, or −0.67 excluding Qwen. Full delivery remains sensitive, however: GPT Sol has the highest mean and an SD of only 1.93, yet its full-credit rate varies by 9.12 percentage points. Both GLM variants have full-credit gaps of 13.50 points.” · Show in paper
Verified
Table 9 and Figure 3(a) directly support both the negative correlation between mean score and SD and the persistent sensitivity of R1.
Bursts That Outlast Themselves: Persistent Degradation in Prefix-Cached Agent Serving· Method
PaperSource excerpt
“We run vLLM 0.20 (ROCm build 0.20.2rc1.dev253) with automatic prefix caching on single AMD MI300X GPUs, one engine per GPU, BF16 weights and KV cache, default scheduler (chunked prefill, FCFS), 40K maximum context, greedy decoding with a fixed output length per turn... Both runs of a pair share one base arrival stream: every base session arrives at the same time with the same identity and shape. The burst run additionally receives an independent stream of extra sessions (rate 1.5λ for 5 minutes, i.e. 2.5× total arrivals) starting at minute 15.” · Show in paper
Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving
PaperCodeSource excerpt
“Implemented on vLLM and evaluated on production-style traces with DeepSeek-V2-Lite, our approach reduces P95 TTFT by upto 81% and raises SLO attainment by upto 79% over state-of-the-art disaggregated schedulers, at sub-millisecond per-request routing cost.” · Show in paper
Verified
The evaluation results in Section 6 (Figures 4-6) and Section 4.4/5.3 support the improvements in P95 TTFT, SLO attainment, and sub-millisecond routing overhead, though Section 9 (Conclusion) states 'upto 71%' SLO attainment improvement rather than 79%.
BridgeKV: Large-Model Prefill and Small-Model Decoding for LLM Inference via KV Cache Translation
PaperSource excerpt
“In 12 of the 30 settings, the selected configuration uses only the first stage of training (Appendix C), so Md decodes with exactly the weights of its fine-tuned baseline, and BridgeKV still scores higher in all 12. In these settings, the gain comes from the stitched cache alone, not from extra trainable parameters on the decode side.” · Show in paper
Verified
Table 9 confirms 12 settings use S1 only, and Figure 3 confirms higher scores for BridgeKV in all of them.
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
PaperSource excerpt
“In this setting, we find that models generally perform worse at this task than when generating patch files; for instance, Claude 2 scores at 2.2% compared to 4.8% in the main table for “oracle” retrieval. Even when controlling for instance length, generating on the shorter half of the task instances by input tokens yields 3.9% compared to 7.8% for generating patches with Claude 2.” · Show in paper
Verified
The reported comparison for Claude 2 (2.2% vs. 4.8% overall, and 3.9% vs. 7.8% on shorter instances) supports the claim.
PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
PaperSource excerpt
“Table 4 shows that the full PagedWeight system achieves the lowest perplexity. Removing routing statistics or prompt residuals degrades perplexity to 7.26/10.13 and 7.31/10.19, showing that both online measurements help. Removing page movement turns PagedWeight into a static mixed-precision plan and further increases perplexity to 7.43/10.33. Removing global sensitivity falls back to uniform quantization and performs worst.” · Show in paper
Verified
Table 4 directly substantiates that removing any single component increases perplexity on both Wikitext2 and C4.
Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving
PaperCodeSource excerpt
“Overall, these results indicate that KV cache compression techniques introduce only limited additional latency for the first token, with the exception of TurboQuant, whose more complex quantization pipeline leads to noticeable overhead.” · Show in paper
Verified
Table 3 and Figure 2 support the claim for the evaluated datasets, though GovReport results for TurboQuant are omitted due to 3-day single-run execution times.
QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs· Method
PaperCodeSource excerpt
“In NVFP4, each fused matrix uses a single tensor-level weight scale and a common input-activation scale. To reproduce this inference-time behavior, QATFactory ties the corresponding scales during training: it computes the shared weight scale from the concatenated projection weights and tracks one activation scale for their common input.” · Show in paper
FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving· Method
PaperCodeSource excerpt
“FlashInfer guarantees both attention and contraction stage are compatible with CUDAGraphs (Gray, 2019; Nguyen et al., 2021). Both attention and contraction stage use persistent kernel and the grid size is fixed once compiled, which means the kernel is launched with the same grid size for each generation step. We set the fixed offset for each section of the workspace buffer to store partial outputs and plan information to make sure the pointers passed to the kernel are the same for each generation step, meeting the requirement of CUDAGraphs” · Show in paper
A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation
PaperCodeSource excerpt
“Dropping the residual entirely (rank-only compression) raises 2× perplexity from 6.54 to 7.27, a gap that additional rank cannot close.” · Show in paper
Verified
Table 11 directly supports both the necessity of the rotated residual and the saturation at 4 bits.
SoScale: Second-Order Scale Search for FP4 Quantization
PaperSource excerpt
“We identify an outlier-induced scale mismatch in hierarchical FP4 quantization: default range-preserving scales can reserve scarce FP4 levels for regions that receive no assignments, reducing resolution for dense weight values.” · Show in paper
Verified
Figure 1a-c directly illustrates the unoccupied extreme levels under default scaling, the reduced element-wise error in the dense region, and the perplexity reduction achieved by SoScale compared to ScaleSearch and RTN.
gpt-oss-120b & gpt-oss-20b Model Card
PaperCodeSource excerpt
“Our models demonstrate smooth test-time scaling. In Figure 3, we sweep over the different reasoning modes of the model (low, medium, high) and plot accuracy versus average CoT+Answer length. We generally see log-linear returns on most tasks, where longer CoTs provide higher accuracy at a relatively large increase in final response latency and cost.” · Show in paper
Verified
Figure 3 clearly demonstrates smooth, monotonic accuracy improvements from low to medium to high reasoning effort on AIME 2025 and GPQA Diamond.
Programming a Coding Agent's Weights by Hand
PaperSource excerpt
“This paper delivers cheap, precise, verifiable control of a coding agent’s decisions by LLM-written weight edits, and a measured account of when such control does and does not raise solve rate.” · Show in paper
Verified
The manuscript thoroughly substantiates both the precise, dose-dependent decision control and the nuanced, negative/unconfirmed solve-rate findings with appropriate controls and pre-registered tests.
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model
PaperCodeSource excerpt
“The rank ordering of configurations is consistent across all three seeds: full fine-tuning > LoRA r=32 ≈ LoRA r=16 > LoRA r=8 > LoRA r=4 > LoRA r=2. Lower ranks show higher variance (std up to 0.063 for r=2), suggesting sensitivity to initialization at minimal parameter budgets. QLoRA configurations are among the most stable, with std of 0.006 (INT8) and 0.015 (NF4), likely because quantization regularizes the optimization landscape.” · Show in paper
Verified
Table 3 and Figures 1–3 directly substantiate the consistency of rank ordering and the reported standard deviations across seeds.
BridgeKV: Large-Model Prefill and Small-Model Decoding for LLM Inference via KV Cache Translation· Method
PaperSource excerpt
“Unless otherwise stated, the main system results use a 200-request trace with 1024 input tokens and 1024 output tokens per request. Request arrivals follow an exponential distribution with an arrival rate of 2.0 requests/s.” · Show in paper
SWE-Bench Pro V2: A Cleaner, Harder-to-Game Coding Benchmark
PaperSource excerpt
“For GLM-5.3 and Kimi K3, accuracy barely moves: only one of their six scaffold pairs differs significantly, Kimi K3’s Pi against OpenCode by 1.7 points. Inkling is the exception, with Pi trailing the other two scaffolds by 5.8–7.5 points.” · Show in paper
Verified
Table 3, Table 4, and Figure 4a directly substantiate the accuracy spreads, p-values, and token-cost differences.
End-to-End Context Compression at Scale
PaperCodeSource excerpt
“In Figure 13, LCLMs achieve the highest accuracy across all compression ratios on GSM8K, with particularly strong gains over baselines at higher compression ratios.” · Show in paper
Verified
Table 9 and Figure 13 directly support the claim across all three compression ratios.