By @keithtyserSourceOpenVersions
Fork
Clone
git clone https://openversions.com/git/keithtyser/programming-a-coding-agent-s-weights-by-hand.git
Cite
Version link
https://openversions.com/keithtyser/programming-a-coding-agent-s-weights-by-hand/commit/59abc3395c7664ae49633e6da6ccaf55979a7c73
Version identifier (commit hash)
59abc3395c7664ae49633e6da6ccaf55979a7c73

Fork to create your own copy with credit to the source. Clone to get a Git copy on your computer.

Claims

Findings

Headline finding

LLM-written closed-form weight edits to a few MLP neurons, packed as standard LoRA adapters, deliver cheap, precise, verifiable control of a multi-step coding agent's decisions, alongside a measured account of when such control does and does not raise solve rate.

The compiled LoRA adapter closely reproduces the intended inference-time neuron-addition hook, achieving residual-stream cosine 0.972–0.996 across layers 18–41 and completion-token KL(hook || adapter) of 1.1×10−41.1 \times 10^{-4} and 1.2×10−41.2 \times 10^{-4}, which is 99 and 110 times smaller than the edit's own effect.

Always-on weight edits move their targeted agent decision monotonically with edit strength on held-out decisions, while matched random-direction controls of the same size do not.

On the 170 development tasks, low-dose always-on explore-more edits (pooled strengths 0.125–0.5) exceed the unedited model by 2.7 tasks over 12 batches (p = 0.030) and exceed matched random controls by 3.6 tasks over 7 shared batches (p < 0.001), while strength 1.0 solves 6 tasks fewer than the unedited model.

Three frontier LLM authors (Claude, GPT, Gemini) all produced working edits under one brief, with Claude's before-first-edit gate solving 30.3 against 25.3 development tasks (+5.0, p = 0.014 uncorrected), though no edit survives Holm–Bonferroni correction and the gate did not separate from its ungated 0.125-gap control (+3.75, p = 0.11).

Neither of two pre-registered held-out tests on 130 repository-disjoint tasks confirmed that the development-set solve-rate gain transfers out-of-sample (always-on explore-more 0.25: +0.33 tasks-equivalent, p = 0.92; before-first-edit gate: +3.0 tasks-equivalent, p = 0.50).

Control of an agent decision is not enough to solve more tasks for any tested direction other than explore-more, including submit-sooner, think-briefly, test-before-submitting, failed-edit neurons, compiled prompt rules, and one-shot gradient weight deletion.

The unedited greedy Gemma 4 E4B agent fails by inconsistency rather than incapacity (solving 22–30 of 170 tasks per run, 51 at least once across 13 runs), and failed file edits (within-task OR 0.42, 95% CI 0.23–0.74) and long exploration of 8+ calls before the first edit (OR 0.12, 95% CI 0.04–0.38) are associated with lost episodes.

Methods

Locate a decision direction dl=μ1−μ0d_l = \mu_1 - \mu_0 as the difference of class means of layer outputs at labelled decision points on a training split, keeping layers with high held-out AUC.

Source excerpts

“For layer l, the direction dl = µ1 − µ0 is the difference of the class means of the layer output on the training split. Its norm kdlk is the class gap. We use this single split, without further regularisation or cross-validation. We keep a direction if projecting held-out decisions onto dl separates the classes, measured as AUC.” · Show in paper

Overwrite the quietest MLP neuron per layer with a constant ridge read-out qq (or gated read-out r−θqr - \theta q) and closed-form down-projection down:,j=±βρnu^/gelu(G)\text{down}_{:,j} = \pm \beta \rho \sqrt{n} \hat{u} / \text{gelu}(G) where u^=unit(dl/wpost)\hat{u} = \text{unit}(d_l / w_{\text{post}}), packed as a standard LoRA adapter.

Source excerpts

“In each layer we overwrite the neuron with the smallest product of mean absolute activation and down-column norm (0.22–0.32 of the layer median).” · Show in paper

“gatej = G q, upj = q, down:,j = ± β ρ √ n u / ˆ gelu(G), uˆ = unit(dl/wpost),” · Show in paper

Construct a matched random control by overwriting the same neurons with the same gate and up rows, and a write direction built from dld_l with random signs per dimension orthogonalised to dld_l by Gram–Schmidt, mapped through 1/wpost1/w_{\text{post}} and normalised to the same down-column norm.

Source excerpts

“The control overwrites the same neurons with the same gate row and up row. We build its write direction in residual space. We take dl with a random sign per dimension and make it orthogonal to dl by Gram–Schmidt. We then map it through 1/wpost and normalise it to unit length, so the down column has the same norm as the edit’s.” · Show in paper

Run a six-step weight-programming loop (Diagnose, Locate, Write, Predict, Verify by replay and agent evaluation, Score) where steps 1–4 run on CPU and only verification uses a GPU, with pre-run written predictions.

Source excerpts

“Figure 1 shows the loop. Algorithm 1 (Appendix A) lists its six steps: diagnose, locate, write, predict, verify by replay and by agent evaluation, and score. The author is an LLM with shell access to the model weights, stored activations and logged episodes. Steps 1–4 run on a CPU; only verification uses a GPU.” · Show in paper