Headline finding
AQLoRA is a zero-search, data-free recipe that uses one CPU pass over pretrained weights to set per-layer precision and adapter placement, achieving speedups over well-tuned QLoRA (11.3 ± 2.3% faster in its speed setting with 0.1 to 1.5 accuracy points lost, and 4.8 ± 2.0% faster in its quality setting at accuracy parity with QLoRA and within a point of fp16 LoRA) across six models and four architecture families (1.4B to 14B) on Commonsense-170K, for a memory premium of 0.06 to 0.8 GiB on 1–4B models, 0.5 GiB at 7B, and 1.9 GiB at 14B.
The accuracy, memory, and speed figures match the reported tables, and the authors appropriately scope the timing claims to the five timed models (1.4B to 7B).
“We evaluate on Commonsense-170K across six models and four architecture families, from 1.4B to 14B. The speed setting trains 11.3 ± 2.3% faster than well-tuned QLoRA and gives up 0.1 to 1.5 accuracy points. It was faster in every one of thirteen independent timing sessions, at worst by 7%. The quality setting trains 4.8 ± 2.0% faster. Its accuracy is level with QLoRA on every model and within a point of fp16 LoRA. The memory premium depends on how the architecture packs its layers: 0.06 to 0.8 GiB on the 1–4B models, 0.5 GiB at 7B, 1.9 GiB at 14B.” · Show in paper