Headline finding
In factual acquisition on an anonymized OpenStreetMap-derived benchmark with Qwen3-4B, QLoRA rank (8, 16, 32, 64) induces an acquisition-retention frontier: low-rank QLoRA preserves out-of-domain (OOD) performance but acquires fewer facts, higher ranks improve same-fact paraphrase generalization at an increasing cost to unrelated benchmarks, and full fine-tuning (FFT) behaves as a conservative baseline that retains general capabilities well without reaching the highest factual-acquisition regime.
The five-seed results in Figure 1 and Table 5 directly support the full claim, including the relative positions of FFT and QLoRA ranks 8, 16, 32, and 64.
“Comparing full fine-tuning (FFT) with quantized low-rank adaptation (QLoRA) at ranks 8, 16, 32, and 64, we find that rank induces a clear acquisition–retention frontier. Low-rank QLoRA preserves out-of-domain (OOD) performance but acquires fewer facts, whereas higher ranks improve same-fact paraphrase generalization at an increasing cost in performance on unrelated benchmarks. FFT behaves as a conservative baseline: it retains general capabilities well, but does not reach the highest factual-acquisition regime.” · Show in paper