Search publications
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“The next lemma is the constructive finite-dimensional Carathéodory argument for preserving an expectation. Its connection with moment matching is classical [3]; we include the exact elimination and its bit bound because both are needed by the algorithm.” · Show in paper
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
PaperCodeSource excerpt
“Only when we choose α from the sweet spot region (0.4-0.6) can we get small quantization errors for both weights and activations, and maintain the model performance after quantization.” · Show in paper
Verified
Figure 10 directly demonstrates the accuracy drop outside the 0.4–0.6 sweet spot for OPT-175B.
AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning· Method
PaperCodeSource excerpt
“For each linear layer ℓ with weights Wℓ, flattened into blocks b of 64 elements (matching bitsandbytes), εℓ = 1/|Wℓ| ∑_b ||b − absmax-NF4^-1(absmax-NF4(b))||_2^2, sℓ = Pr[|Wℓ,ij| < τ], with τ=0.01 and the true 16-level NF4 quantile grid. The pass runs on CPU over the fp16 checkpoint, is data-free, and requires no forward pass.” · Show in paper
Resource-Efficient Speculative Decoding for Long-Context LLM Serving
PaperSource excerpt
“In Fig. 7, streaming access for the full query group yields the largest throughput gain. With 16K inputs, K2 improves throughput over K1 by an average of 83.2% on Qwen3 and 117.1% on InternLM2.5.” · Show in paper
Verified
The ablation results in Figure 7 and Section 5.3.1 directly substantiate the individual and cumulative gains of streaming verification.
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
PaperCodeSource excerpt
“As shown in Table 3, SmoothQuant can match the FP16 accuracy on all evaluation datasets with all quantization schemes. LLM.int8() can match the floating point accuracy because they use floating-point values to represent outliers, which leads to a large latency overhead (Table 11). The W8A8, ZeroQuant, and Outlier Suppression baselines produce nearly random results, indicating that naively quantizing the activation of LLMs will destroy the performance.” · Show in paper
Verified
Table 3 provides direct empirical evidence that SmoothQuant-O1/O2/O3 closely match FP16 accuracy and perplexity on OPT-175B while the three baselines degrade to near-random performance.
Bursts That Outlast Themselves: Persistent Degradation in Prefix-Cached Agent Serving
PaperSource excerpt
“Table 2 lists all completed pairs: in 9 of 12 only the burst run degraded persistently, in 0 only the no-burst run did, 2 degraded in both arms and 1 in neither (two-sided sign test over discordant pairs, p = 0.00391).” · Show in paper
Verified
The paired design, statistical test, and control experiments cleanly establish history-dependent persistent degradation on vLLM.
OpenAI Math GitHub Repository
PaperCodeSource excerpt
“Proposition 4.2 (Uniform scheduling bound). There is one polynomial p, independent of the instance, such that preprocessing uses at most p(L) bit operations and serving request t uses at most p(L + ⌈log2(t + 1)⌉) bit operations, for every reachable history and internal state. Activation occurs at a finite deterministic time t∗ depending only on (n, k, d).” · Show in paper
Verified
The capped simulation schedule and output-size bound S = O(g(t)+1) cleanly link the constructor's arbitrary finite runtime T_P to the uniform polynomial per-request bound.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“Lemma 12.4 (Bounded-displacement matching). Let Λ be a full-rank lattice in R^2 and let T be invertible with |det T| = 1. There is a bijection ψ : Λ → Λ and a finite bT such that |ψ(z) − Tz| ≤ bT (z ∈ Λ).” · Show in paper
ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs
PaperCodeSource excerpt
“Table 2 shows that conventional PTQ methods yield diminishing returns on NVFP4. Notably, QuaRot (Hadamard transform) results in performance regression compared to RTN on Llama 3.1-8B, which confirms our motivation (Figure 2) that rotation disrupts the isolation benefits of fine-grained blocking. Similarly, SmoothQuant offers marginal gains due to the limited capacity of 4-bit weights. In contrast, ARCQuant consistently achieves the best results in all metrics, effectively suppressing outlier errors in the activation matrices.” · Show in paper
Verified
Table 2 confirms that NVFP4+QuaRot regresses on Llama 3.1-8B (70.18 avg, 6.99 PPL vs 70.45 avg, 6.95 PPL for RTN), while ARCQuant achieves the highest average accuracy and lowest PPL on both models.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“Repeated projection reduces dimensions j ≥ 4 to dimensions two and three. In the remaining dimensions, the sampling argument uses independent Poisson batches. A moment argument controls the size of the decoded set; its geometric input is a rich-line estimate with the field characteristic kept larger than the relevant polynomial degrees.” · Show in paper
End-to-End Context Compression at Scale
PaperCodeSource excerpt
“Training at scale confirms the choices from the small-scale sweep: both the continual pre-training losses (Figures 9 to 11) and the downstream ablations (Appendix F.6) again favor an encoder window W = 1024 and causal masking. At scale, the attention-based adapter is competitive with the MLP adapter; we keep the MLP adapter, which adds less computation. The choice of pooling operator depends on the compression ratio: mean pooling has a small edge over concatenation at 16x compression, while concatenation slightly outperforms mean pooling at 4x.” · Show in paper
Verified
Both the from-scratch pre-training sweeps and the full-scale continual pre-training and downstream ablations support these architectural conclusions.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“Procedure \mathcal{L}(A, D, a, b). Execute the first applicable base case. If neither applies, execute Steps 3–6 in order.” · Show in paper
AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning· Method
PaperCodeSource excerpt
“Given a protection budget ρ ∈ [0, 1] (sensitive-frac), the K=⌈ρL⌉ layers with largest εℓ are loaded in fp16; all others in NF4 with double quantization; lm_head stays fp16.” · Show in paper
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model
PaperCodeSource excerpt
“Our results show that LoRA with r=16 recovers within 11.6 percentage points of full fine-tuning accuracy (59.6% vs. 71.2% exact-match) while training fewer than 1% of parameters and consuming 31% less peak GPU memory.” · Show in paper
Verified
The numbers in Table 2 and the Pareto plots in Figure 1 directly support the accuracy, parameter count, and memory reduction claims for seed 42.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“On a closing edge, its input and required endpoint are known. A butterfly updating each coordinate at most once has at most one route between those endpoints; the decoder fills the unknown bits along that route with the forced values, rejecting inconsistencies.” · Show in paper
AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning
PaperCodeSource excerpt
“Under that control the ranking has no measurable advantage (Appendix N lists every setting). Across five models and three budgets, top-K beats the median matched draw in 11 of 15 settings, and 50 of the 150 draws beat top-K where chance predicts 75. A sign test does not separate that from chance (p=.12), and the median gap is 0.01 perplexity.” · Show in paper
Verified
The parameter-matched control in Table 19 and downstream comparisons in Section 7.2 directly support this negative result.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“From the filtered lists, enumerate only pairs whose signed reduced weights add to T F as integers and whose signed unreduced weights do not add to t F s as integers. Both sums count mask positions with their multiplicities, including a position selected by both masks.” · Show in paper
Norm-indexed sparse Bernoulli perturbation using geometric keys .
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“f(n) = ‖qy˜(n)‖2 2 , κ(n) = ( (V (yi(n)))D i=1, ⌊f(n)⌋ ) , 𝒦 = {κ(n) : n ∈ G}.” · Show in paper
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
PaperCodeSource excerpt
“At ≈ 2.2 bit (group-size 128; using FP16 scale and 2-bit zero point per group) the perplexity increase is already less than 1.5 points, while dropping to 0.6 - 0.7 at ≈ 2.6 bit (group-size 32), which is only slightly worse than vanilla 3-bit and might be interesting for practical kernel implementations. Further, if we reduce group size to 8, we can apply ternary (-1, 0, +1) quantization, which achieves 9.20 WikiText2 PPL on OPT-175B, a less than 1 point drop.” · Show in paper
Verified
Table 7 directly supports the 2-bit grouped perplexity numbers; the ternary group-size 8 result (9.20 PPL) is reported in the text.
Predecessor-span localization via replica likelihoods and discrete smaller-ball coding.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“Lemma 11.5 (A discrete description of a smaller ball). There is a discrete code C, determined by (A∗, sI), and an A′ = (A∗, C)-measurable ball B(c′, r′) supporting the conditional signal law” · Show in paper