Search publications
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
PaperSource excerpt
“In this setting, we see increases in performance, with GPT-4 jumping from 1.3% to 3.4% and Claude 2 from 4.8% to 5.9%.” · Show in paper
Verified
Table 6 and Table 18 confirm the performance gains when collapsing unedited context in the oracle setting.
Bursts That Outlast Themselves: Persistent Degradation in Prefix-Cached Agent Serving· Method
PaperSource excerpt
“We replay whole-session shapes—initial context, per-turn output, per-turn context growth and inter-turn gaps—of 200 trajectories sampled from public mini-SWE-agent leaderboard runs on SWE-bench Verified by six 2026 models” · Show in paper
MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference
PaperSource excerpt
“Moment-informed eviction (MI) alone yields +0.57 at L=128, while normalization-corrected inference (NC) alone contributes +2.41, confirming that restoring the evicted output direction is more impactful than refining selection. The ~4x ratio is consistent with the analysis in Section 3, where the dominant error source is the amplification coefficient \gamma(\theta) that only NC addresses. Combining both achieves +3.17, exceeding the sum of individual gains by +0.19 due to the reinforcing loop described in Section 4.2.” · Show in paper
Verified
Table 3 confirms both the ~3-4x contribution ratio of NC over MI and the super-additive combined gain at all three tested budgets.
ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs
PaperCodeSource excerpt
“On Llama 3.1-70B, ARCQuant consistently improves over the NVFP4+RTN baseline across all reported metrics, with gains of 0.65 in average zero-shot accuracy, 0.23 lower WikiText2 perplexity, and 0.50 higher MMLU, while maintaining a competitive margin to the FP16 baseline. We also validate ARCQuant on the MoE model Mixtral 8x7B-Instruct, where it again achieves consistent improvements over NVFP4+RTN, remaining close to FP16 performance.” · Show in paper
Verified
Table 9 confirms the exact reported gains on Llama 3.1-70B and Mixtral 8x7B-Instruct.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“Process the subgroup coset systems successively. Within each coset whose current mass exceeds its uniform mass, multiply all current point masses by the ratio of the cap to the current mass.” · Show in paper
MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference
PaperSource excerpt
“Figure 2(d) confirms the amplification empirically: a network-wide average mass below 0.08 produces over 20% relative error on average, with peaks above 60% in the shallowest layers.” · Show in paper
Verified
Figure 2(a)-(d) directly reports these exact averages and peak values.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“We prove both propositions by an averaging estimate that retains cap components as complete sampling units. This avoids replacing a whole-cap trace estimate by a pointwise one. The two-sided level averaging argument is a local form of the quantitative Parisi comparison technique in [3, Lemma 3.3].” · Show in paper
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“Applying Lemma 5.1 and then taking the supremum over the parent edges yields L 2 m ≤ L 2 m+1 − 1/12. Summing over m gives D2 ≥ L 2 k ≥ L 2 0 + k/12 ≥ 1 + k/12.” · Show in paper
FP4Quant: Efficient 4-Bit Communication Compression for Model-Parallel LLM Training
PaperSource excerpt
“Overall, FP4Quant enables efficient low-bit communication with substantial acceleration under TP while incurring negligible runtime overhead under PP, demonstrating its effectiveness across diverse parallel configurations.” · Show in paper
Verified
Table 2 supports both the TP speedup and the negligible overhead under PP.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“Write K = |Σ|, put B = 2K + 2, and assign the symbols distinct odd digits c(a) ∈ {1, 3, . . . , 2K − 1}. Define C(a1, a2, . . .) = ∑k≥1 c(ak)/Bk” · Show in paper
Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving
PaperCodeSource excerpt
“Claim 2: prefill compute is not the bottleneck for tail TTFT. Table 3 decomposes P95 TTFT into its four constituents. Prefill execution accounts for only 2-23% of P95 TTFT and the remaining 77-98% is queue wait and KV-cache transfer.” · Show in paper
Verified
Table 3 provides the exact breakdown across all five workloads, confirming that prefill execution is 2-23% of P95 TTFT.
AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning
PaperCodeSource excerpt
“We evaluate on Commonsense-170K across six models and four architecture families, from 1.4B to 14B. The speed setting trains 11.3 ± 2.3% faster than well-tuned QLoRA and gives up 0.1 to 1.5 accuracy points. It was faster in every one of thirteen independent timing sessions, at worst by 7%. The quality setting trains 4.8 ± 2.0% faster. Its accuracy is level with QLoRA on every model and within a point of fp16 LoRA. The memory premium depends on how the architecture packs its layers: 0.06 to 0.8 GiB on the 1–4B models, 0.5 GiB at 7B, 1.9 GiB at 14B.” · Show in paper
Verified
The accuracy, memory, and speed figures match the reported tables, and the authors appropriately scope the timing claims to the five timed models (1.4B to 7B).
OpenAI Math GitHub Repository
PaperCodeSource excerpt
“For either T = EL or T = RL, (1 − r2L)Id ≤ T ∗T ≤ Id. (4.10) In addition, let Fe apply F(z) to the port register, controlled on z. Then ∥FeEL − RLV ∥ ≤ 2τ. (4.11)” · Show in paper
Verified
The telescoping Gram-matrix argument, the truncation error bound, and the Newton's identities multiset argument are all rigorous.
OpenAI Math GitHub Repository
PaperCodeSource excerpt
“Let P = f\sigma_d be a probability with a nonnegative Borel density f \le e^b, where b \ge 0. Let S \sim P, let A be an independent k \times d standard Gaussian matrix, and put Y = AS. Let a finite message W be drawn from a kernel of (A, Y), conditionally independently of S. Let B be a p \times d standard Gaussian matrix independent of the entire joint experiment producing (S, A, Y, W). Then I(S; W \mid B, BS) \le H(W)/q + C(d + b/d) for an absolute C, uniformly over the prior and message kernel.” · Show in paper
Verified
The proof in Section 5 is complete, with explicit integrability and measurability checks.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“N(d) = X e∈B exp(de)cecTe , f(d) = log det N(d) − 1/q X e∈B de. (4.2)” · Show in paper
Bursts That Outlast Themselves: Persistent Degradation in Prefix-Cached Agent Serving
PaperSource excerpt
“The regime appears at loads the cache-warm server sustains, is absent without prefix caching, and also appears with Qwen3-32B on SWE-bench agent trace shapes and on SGLang, which tolerates larger triggers.” · Show in paper
Verified
The SGLang paired runs and 32B trace-shape runs support the generalization, and the authors appropriately note that 32B provides weaker evidence of history dependence than 8B.
Learning New Facts with QLoRA: An Acquisition-Retention Frontier
PaperCodeSource excerpt
“Figure 2 shows that configurations with stronger OOD degradation also exhibit larger drift from the pretrained model. Higher-rank QLoRA checkpoints have larger symmetric KL divergence and larger effective dense update magnitudes. The strongest forgetting regime, QLoRA r=64, also has the largest SVD intruder excess, indicating a larger change in the leading spectral structure of adapted weight matrices.” · Show in paper
Verified
Figure 2 clearly shows monotonic increases in symmetric KL, dense RMS drift, and SVD intruder excess with QLoRA rank, exceeding the FFT baseline.
Binary-multiplicity truncation and rounding reduction to an explicit multigraph of polynomial size.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“For n ≥ 2, let cuv be the binary multiplicity of the unordered pair uv and set c′uv = min{cuv, k}, auv = ⌊ 2n2 c′uv / k ⌋. (6.6)” · Show in paper
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“Represent the independent weights as τe = Φ(Ue), Ue = ∑_{h≥1} 2^{−h} , where all are independent Bernoulli variables of parameter 1/2.” · Show in paper
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“We will keep one copy of g∗ throughout: first remove the product masks, then eliminate the other linear factors, and finally identify the sampling law of the surviving cube root.” · Show in paper