Headline finding
In the T5-small (60M-parameter) and WikiSQL setting, LoRA with r=16 on {q,v} is the Pareto-optimal configuration for unconstrained memory, recovering within 11.6 percentage points of full fine-tuning exact-match accuracy (59.6% vs. 71.2%, or 83.7% of full fine-tuning accuracy) while training fewer than 1% of parameters (589,824 parameters, 0.97%) and consuming 31% less peak GPU memory (1.60 GB vs. 2.31 GB).
The numbers in Table 2 and the Pareto plots in Figure 1 directly support the accuracy, parameter count, and memory reduction claims for seed 42.
“Our results show that LoRA with r=16 recovers within 11.6 percentage points of full fine-tuning accuracy (59.6% vs. 71.2% exact-match) while training fewer than 1% of parameters and consuming 31% less peak GPU memory.” · Show in paper