Search publications
OpenAI Math GitHub Repository
PaperCodeSource excerpt
“Lemma 7.3 (Well-defined construction and query cap). One can choose w > 0, depending only on K0, t and the fixed Picard counts, and then choose the interpolation orders, so that the entire construction is well defined for all sufficiently large d. Every routine reached from a top call satisfies (7.4). Its total number of seed slots and oracle queries is at most polylog(d)δ^−t on every execution.” · Show in paper
Verified
The bottom-up induction for shift norms and the top-down height bound are noncircular and rigorously proved.
MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference· Method
PaperSource excerpt
“The eviction score combines the attention weight with the residual norm as score(j) = \alpha_j \cdot ||r_j||, and the token with the lowest score is evicted first.” · Show in paper
TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate· Method
PaperSource excerpt
“Therefore the quantizer Qmse : R^d → {0, 1}^{b·d} first computes Π · x and then computes and stores the indices of the nearest centroids to each coordinate of this vector. The dequantization map _mse : {0, 1}^{b·d} → R^d reconstructs the vector by retrieving the centroids corresponding to the stored indices and then rotating the result back to the original basis through multiplication with Π^⊤.” · Show in paper
MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference· Method
PaperSource excerpt
“Second, we apply a simple numerical safeguard: after computing \tilde{S}, we clamp any entry whose magnitude falls below a threshold \tau = 10^{-6} to zero, preventing noise in the low-significance bits from propagating through the matrix-vector product \tilde{S} q.” · Show in paper
Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention
PaperSource excerpt
“We instead combine Root’s shape-dependent coefficients with p99 outlier clipping and execute the NS matrix products directly in NVFP4. In our numerical comparison, the relative L2 error decreases from approximately 10% for a plain all-FP4 route to approximately 3% for the proposed route.” · Show in paper
Verified
The numerical error reduction and isolated RTX 5090 speedups over BF16 and GRASP are directly supported by Figure 3b, Table 6, and Appendix E.
QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs· Method
PaperCodeSource excerpt
“the original high-precision pretrained checkpoint serves as a frozen teacher to supervise a low-precision student initialized from the same weights. Given an input sequence of length L, the framework minimizes the token-averaged forward Kullback–Leibler (KL) divergence between the teacher distribution pT,t and the student distribution pS,t over their shared vocabulary V” · Show in paper
OpenAI Math GitHub Repository
PaperCodeSource excerpt
“A single uniform randomized online algorithm preprocesses these data in at most p(L) bit operations and serves request rt, for every reachable history and internal state, in at most p(L + ⌈log2(t + 1)⌉) additional bit operations. It uses only unbiased random bits and moves exactly one chosen server to each request, permitting a zero-length move. For every fixed finite request sequence σ independent of those bits, E costA,s(σ) ≤ C(log(k + 1))2 OPTs(σ) + Bd,k,s, (1) where Bd,k,s is finite and independent of σ and its length. The algorithm does not need to know that length in advance.” · Show in paper
Verified
Assuming the companion existence theorem (Theorem 2.1), the proof of Theorem 1.1 rigorously combines the bit-complexity schedule and the competitive-ratio suffix analysis.
M2XFP: A Metadata-Augmented Microscaling Data Format for Efficient Low-bit Quantization
PaperCodeSource excerpt
“Evaluation results demonstrate that our method substantially narrows the accuracy gap, achieving on average a 70.63% reduction in accuracy loss compared to MXFP4 and a 37.30% reduction relative to the latest NVFP4 on LLM benchmarks. Furthermore, our design delivers up to 1.91× speedup and 1.75× energy savings over state-of-the-art accelerators.” · Show in paper
Verified
The reported 70.63% and 37.30% accuracy loss reductions match the Table 2 averages, and the 1.91x speedup and 1.75x energy savings are supported by the normalized latency and energy comparisons in Figure 13.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“We use the standard chain rule for relative entropy [8, Theorem 2.5.3]:” · Show in paper
OpenAI Math GitHub Repository
PaperCodeSource excerpt
“Lemma 10.3 (One policy for all finite inputs). For every metric space X and every distinct tuple s ∈ X^k, there is a single randomized causal policy A_s such that E cost_{A_s,s}(σ) ≤ α OPT_s(σ) for every finite request sequence σ in X. (10.13) For each fixed input its decisions and cost are measurable functions of its internal randomness. Neither its rule nor its randomization depends on an eventual horizon.” · Show in paper
Verified
The compactness argument and measurability verification on with Lebesgue measure are complete and rigorous.
QLoRA: Efficient Finetuning of Quantized LLMs· Method
PaperCodeSource excerpt
“The information theoretically optimal data type for zero-mean normal distributions with arbitrary standard deviations in the range is computed as follows: (1) estimate the quantiles of a theoretical distribution to obtain a k-bit quantile quantization data type for normal distributions, (2) take this data type and normalize its values into the range, (3) quantize an input weight tensor by normalizing it into the range through absolute maximum rescaling.” · Show in paper
Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention· Method
PaperSource excerpt
“For the first moment mt, Hadamard mixing smooths the distribution before SR and adaptive 4/6 quantization. For vt, we use an ordered three-stage preprocessing pipeline before NVFP4 storage: (1) square root shortens the tail; (2) a BF16 tile mean separates the positive common component and leaves an approximately zero-mean residual; and (3) a 16-element Hadamard transform spreads localized energy and makes the residual more symmetric. ... Reconstruction inverts the transform and mean separation and then squares the result.” · Show in paper
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model
PaperCodeSource excerpt
“QLoRA INT8 and NF4 configurations achieved 52.8% and 53.2% exact-match accuracy respectively, compared to 53.4% for the FP16 LoRA r=8 baseline with the same {q,v} module set. The accuracy cost of quantization is therefore minimal (0.2–0.6 points), while the memory savings are substantial: both QLoRA configurations require only 0.60 GB peak training memory, a 62% reduction relative to FP16 LoRA (1.59 GB) and a 74% reduction relative to full fine-tuning (2.31 GB). Inference latency is higher for INT8 (921 ms/query) than NF4 (614 ms/query) and FP16 LoRA (around 380 ms/query), reflecting the overhead of de-quantization on a T4 GPU. NF4 represents the best quantization trade-off: near-FP16 accuracy at 0.60 GB memory with 34 tok/s throughput.” · Show in paper
Verified
Table 2 and Section 5.5 support the accuracy, memory reduction, and latency comparisons.
SoScale: Second-Order Scale Search for FP4 Quantization
PaperSource excerpt
“Across both Qwen3 models and all three FP4 formats, a SoScale variant obtains the best average score in every model-format group, preserving 96.44%–97.17% of the BF16 average accuracy under MXFP4, 98.99%–99.20% under NVFP4, and 98.38%–99.40% under HiF4. Compared with the strongest non-SoScale baseline in each group, the best SoScale variant improves the average score by 0.21 to 0.77 percentage points; relative to local ScaleSearch, the gains are 0.47 to 1.10 points.” · Show in paper
Verified
The reported relative BF16 percentages and improvements over the strongest non-SoScale baselines and ScaleSearch match the exact values in Table 2.
BridgeKV: Large-Model Prefill and Small-Model Decoding for LLM Inference via KV Cache Translation· Method
PaperSource excerpt
“All stitching layers of a pair have the same shapes, so the batched implementation stacks their weights and runs the whole module in one pass.” · Show in paper
ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs
PaperCodeSource excerpt
“Although tailored for the fine-grained NVFP4 format, ARCQuant demonstrates robust extensibility to INT4 and MXFP4 formats. On Llama 3.1-8B, our method consistently outperforms RTN baselines across all metrics as detailed in Appendix B. Notably, ARCQuant reduces INT4 perplexity by 0.89 and improves zero-shot accuracy by 2.12%.” · Show in paper
Verified
Table 7 confirms that ARCQuant improves INT4 average accuracy from 65.78 to 67.90 (+2.12%) and reduces PPL from 8.84 to 7.95 (-0.89), and also improves MXFP4.
OpenAI Math GitHub Repository· Method
PaperCodeSource excerpt
“The switching experiment. Initially assign the output Hs = ys to each leaf s ∈ LN. Switch these outputs one at a time to Hs = b, proceeding from right to left. At each stage calculate every other output by (4.1).” · Show in paper
OpenAI Math GitHub Repository
PaperCodeSource excerpt
“Theorem 9.8 (Error bounds for the actual estimators). For every allowed task, with precision p defined in Equation (7.8), the actual row outputs, including the pre-incoming scores and support witnesses, satisfy du,a(H, ¯H) ≤ Cb2−p. The clipped vectors have the corresponding ordinary error bound. In particular, every allowed reward query satisfies 1/N Eσ ∑_{x∈Vl} (Xu(x)/u) |¯Wl(x) − Wl(x)|^s ≤ C_b^s 2^{-sp}.” · Show in paper
Verified
The non-circular order of constant choices (Section 12.1) and the avoidance of independence assumptions between child errors are handled carefully.
OpenAI Math GitHub Repository
PaperCodeSource excerpt
“Lemma 8.1 (Rational compilation). In polynomial time one can compute nonnegative rational numbers and l_i such that \ge ^0, \sum_{i<j}( - ^0) \le 1, 0 \le l_i \le \mu_i, L := \sum_i l_i \ge 1 - 10^{-6}. Their binary encoding lengths are polynomial. Every positive l_i is at least an inverse polynomial in the input size, with the polynomial depending only on C.” · Show in paper
Verified
The box-subdivision and term-by-term integration argument is constructive and polynomial-time.
QLoRA: Efficient Finetuning of Quantized LLMs· Method
PaperCodeSource excerpt
“More specifically, Double Quantization treats quantization constants of the first quantization as inputs to a second quantization. This second step yields the quantized quantization constants and the second level of quantization constants . We use 8-bit Floats with a blocksize of 256 for the second quantization as no performance degradation is observed for 8-bit quantization, in line with results from Dettmers and Zettlemoyer [13].” · Show in paper