What we learn along the way

Notes on engineering, research and building a company.

← All articles

We asked an AI to write weights. Writing code worked better.

Can an AI read, repair and write model weights without a training loop? We tested weight-space programming on a ternary model, then compared the result with ordinary code.

Chapters

When a model performs poorly, we usually look at its training or the code that runs it. Could another AI inspect the weights, diagnose the problem and repair them directly? Could it go further and write new weights from scratch?

That is the appeal of weight-space programming: treating a neural network's weights as something a language model can read and write, rather than leaving their construction entirely to a training loop. The MIT CSAIL post Sparks of Weight-Space Programming explores that possibility across reading, editing and writing tasks.1

We tested the idea on our ternary Connect Four model. It fits in about 1.6 MB, and its main weights take just three values: −1, 0 and +1. The game gives us an exact way to judge each move.

We had one model trained with ternary weights and another converted from floating point after training, which had lost much of its playing strength.2

We asked Claude Fable 5.1 to read the pair, repair the weak file and write a replacement block. It found real structure in the weights. Turning that structure into a reliable diagnosis or a strong player was harder. The most useful control came last: asked to write an ordinary program instead, the same model produced a much stronger opponent.

Read the weights, without running the player

The reading test presented two files with identical architecture, tensor shapes and format. One had been trained with ternary weights in the loop; the other had been trained in floating point and converted afterward. We scored games as one point for a win and half a point for a draw. Against the depth-4 search bot, the models earned 93.0% and 12.8% of the available points, respectively. These are separate checkpoints, not the same trained file before and after a shuffle.3

The reader did not know which was which. It was not told the game or given evaluation data. It had no shell and could inspect tensors with a restricted Python tool, including computations on individual tensors, but could not run a forward pass: the calculation that takes a board through the network to its move scores. The weights were inspected in pieces through tools, not poured into one enormous prompt.

In every one of six attempts, Fable called the strong trained file broken. Its confidence ranged from 0.60 to 0.85. One explanation described the trained file's structure as “planted rank-1 components”. It had found a pattern, then supplied a story about how that pattern must damage the computation.4

We changed the presentation without changing the player. Hidden units, attention heads and head dimensions were consistently reordered inside the trained network, with the corresponding weights and scales moved together. Think of renumbering intermediate variables while updating every reference to them. The function stays the same even though its arrangement on disk changes. A parity check returned the same move on all 300 tested positions.

Now Fable called the other file broken in all six attempts.

Six of these twelve attempts were excluded for breaking the sandbox’s tool-use rules, such as importing a disallowed module or reading a tensor outside the supplied loader. The records mark these attempts as void: we report their explanations, but do not count them as valid trials. None attempted a forward pass. Counting only valid attempts leaves three wrong diagnoses for the original pair and three correct diagnoses for the shuffled pair. Including the excluded attempts gives the same direction, not additional valid trials.3

Two panels. A: real ternary weights, 16 hidden units by 32 inputs of the trained network's first feed-forward matrix, beside the same crop of its shuffled copy. Units 0 and 3 swap rows; units 4 to 15 move a quad at a time to rows 48 to 99, swapping places inside their quad, and other units fill their rows. 300 of 300 test positions get the same move from both files. B: all six attempts called the trained file broken, confidence 0.60 to 0.85; after the shuffle all six called the convert-only file broken. Six were void for breaking the sandbox’s tool-use rules; counted only, 0 of 3 right, then 3 of 3. Sanity pair: 6 of 6 found the exact shuffle, 6 still called the copy broken, 5 blamed rotary encoding. All six sanity attempts are void for breaking the sandbox’s tool-use rules, reported rather than counted; none tried a forward pass.

The same player, a different verdict: shuffling the trained file's hidden units moves real trits to other rows without changing a move in the parity check, and all six diagnoses switch sides. For the first feed-forward matrix alone any order of hidden units would do, but the same order must be legal on the next matrix's input side, so it is a structured permutation: whole groups of 128 move to groups, whole quads to quads inside a group, and units swap places inside their quad. Real trits from the files; counts from the reading-test records, with voided attempts reported, not counted.

One explanation from an excluded attempt in the shuffled-pair test focused on how often two layers held the same weight value at the same position. It described the convert-only file's high agreement as evidence of a defect and the shuffled trained file's near-chance agreement as normal. The figure below compares that statistic across all three files: the layout changes, while the shuffled player keeps the same tested moves.3

Same-position trit agreement between pairs of the eight body blocks, per matrix kind, for three files. The trained file's layers agree on 38.2 to 59.8% of positions, the convert-only file's on 39.8 to 65.8%, against 34.4% by chance. The same trained file after the function-preserving shuffle sits at 34.0 to 36.9%. The displayed quotes are from one voided control-pair attempt: it cited high agreement in the convert-only file and called chance-level agreement in the shuffled trained file normal. These are reported explanations, not counted results.

The displayed quotes come from an attempt excluded for breaking the sandbox’s tool-use rules and are reported, not counted. The statistic is a property of the layout. Layers in both original files agree well above chance; the shuffle that leaves every tested move unchanged brings the trained file down to about chance. Computed from the files; why the original layers agree has not been established.

The diagnosis followed the layout, not the function the weights computed. None of the twelve explanations identified conversion after training as the cause of the weak file's failure.

It found the exact shuffle, then blamed a feature we did not use

We then made a simpler comparison: the trained model against its own shuffled copy. Both implement the same player, so neither is broken. The answer format nevertheless forced a choice. All six attempts broke the sandbox’s tool-use rules, so none counts as a valid trial. None attempted a forward pass. We report their answers separately.5

Fable recovered the exact permutation in all six attempts, including the rearrangement of attention heads, dimensions and feed-forward units. That is a substantial piece of reverse engineering. It still called the shuffled file broken every time.

Five explanations blamed rotary position encoding, a method transformers can use to represent position. One said that “scrambling the within-head Q/K dimension order breaks rotary position embedding”. Our model does not use rotary position encoding. It uses learned absolute positions. The consistent query/key shuffle preserves the relevant dot products.5

The forced choice matters: this test did not let the reader answer “neither”. But the invented mechanism matters too. The reader correctly recovered how the files were related, then interpreted that relationship through an architectural assumption that did not apply. Recognising structure and understanding what it computes were different achievements.

Repairing a conversion is not the same as finding a damaged part

For the repair test, we removed the mystery. Fable was told the architecture, the input/output contract and what had happened: a floating-point model had lost its play when converted to ternary. It could edit up to 1% of the trits and make 50 logged practice calls, using practice positions or games against a depth-2 bot. The final evaluation remained separate.6

Two of three attempts submitted the original file unchanged. They investigated possible local defects and tried edits, but the changes they could test were neutral or harmful. Reverting them was a reasonable engineering decision.

The third identified the problem as “distributed quantization noise”. It changed 59 588 trits. Its final game score against the depth-4 bot was 11.5%, compared with the original 12.8%. A small improvement in its practice games had not carried over to the stronger opponent.

Our earlier recovery run, which put the ternary format into training for 12 000 steps, reached 88.5%. That is a useful reference, not a matched-cost contest: the repair attempts had a restricted edit budget and a different feedback process.

The attempted local fixes did not recover the lost playing strength. Within this edit budget and practice allowance, editing the weights did not replace training for the ternary representation.

Write the block, or write the program?

The writing test asked a more constructive question. We kept the trained input embedding that encodes the board and the scoring head that scores the possible moves. Between them, we replaced the network’s eight processing blocks with one block whose weights Fable had to generate. This was not an entirely untrained network.7

The replacement block contained 786 432 ternary weights, plus the block's small floating-point parts. Fable wrote code that generated these values. It did not type them all out. It fitted individual rows to designed targets using least squares and coordinate descent, but did not train the network on game positions or results.

Beside each writer we ran a scripted twin: the same language model working on the same task, with the same limits and practice allowance, but asked to produce Python code instead of weights. Its policy.py evaluator could examine its own move and the opponent’s response, at most two plies ahead. Its mean evaluation time had to stay within 5 ms per position.

Three panels. A: the network the writers completed: 44 context bytes, the trained byte embedding and positions, one written block in place of eight trained ones, and the trained scoring head. The block's four ternary matrices are drawn to scale, 256 by 768, 256 by 256, 256 by 1024 and 1024 by 256: 786 432 trits plus 3 328 small floats. The best writer's first feed-forward matrix is 74.7% plus ones and 0.3% minus ones. B: the twin's policy.py tries each legal move and each legal opponent response, then scores the position, in 0.14 to 0.33 ms against a 5 ms cap. C: of 50 practice calls, the writers used 6, 3 and 3, the twins 8, 11 and 21.

Two ways to build an opponent: a written block between a trained embedding and a trained scoring head, or a short Python program. Both had 50 practice calls and free self-tests on typed positions; only the writers worked under the guard on chaining network operations. The construction notes are the best writer's own description; shapes come from the protocol and the call counts from the run records.

The weight writers did meaningful construction work. They designed attention heads to read rows and diagonals and feed-forward units intended to detect immediate winning moves and opportunities to block the opponent. They fitted ternary rows against the fixed input vectors and wrote outputs along directions the retained scoring head could read. This was an attempt to compile game logic into a transformer, not a random weight generator.

The twins built compact board representations, checked immediate wins and threats, and combined shallow look-ahead with a static evaluation. Each also wrote a solver to check its own reasoning during development. The submitted player still had to obey the two-ply limit.

The two readings of the writer results

Before looking at their playing strength, there is a protocol problem to account for. All three weight writers were void under the original automatic guard. Each made one to three refused steps comparing directions associated with the fixed head's key and value matrices.

Before the final evaluation, we reviewed all 110 flagged steps and found no forward pass. The refused operations compared interface directions rather than chaining the network’s layers on a board. Before seeing the final scores, we retained the original rules as the primary reading and added a second reading that admitted the writers’ results.8

Both belong in the result:

  • Primary, as preregistered: all three weight writers are void. Their files were measured and reported, but their scores do not count as valid writer results. All three scripted twins pass.
  • Second reading: the writers count. One is a partial result and two fail the writing test. All three scripted twins still pass.

The written-block scores below use the second reading. They are not three valid successes under the original protocol, nor does the primary reading establish that all three writers failed on capability.

The code made the stronger opponent

The best written block preserved the game-theoretic value on 86.2% of held-out positions. In other words, it often chose a move that did not turn a win into a draw or a loss, or a draw into a loss. That is less demanding than consistently winning a whole game. The converted file preserved value on 86.0%, so the written block was roughly level on this measure; its gain appeared in games against the search bots.

It scored 21.3% against the depth-4 bot, above the converted file's 12.8% and far below the trained model's 93.0%. The other two written blocks were below the centre-column rule on value preservation. Their depth-4 scores were 4.5% and 1.5%.

The best scripted twin scored 89.8% against the depth-4 bot and 51.0% head-to-head against our trained network. It played roughly level with that trained opponent in this test. The other two twins scored 43.0% and 22.5% head-to-head, so “all three passed” does not mean all three matched the trained model. The pass criterion required both value preservation and a sufficient score against a depth-1 bot, not parity with the trained network.7

The scoreboard, with the rules attached

Each game comparison uses 200 games with colours alternating and a 5% random-move rate for both players. Position-level results use 17 325 held-out positions. These conditions matter: the scores describe this test, not every possible opponent or style of play.9

PlayerHow it was madeDepth-4 scoreAgainst trained
Trained T34Ternary in the training loop93.0%51.3% (self-play)
Training recovery12 000 further training steps88.5%Not measured here
Convert-only T34Float training, then conversion12.8%2.5%
Submitted repair59 588 trits edited11.5%1.5%
Other two repair attemptsOriginal file, unchanged12.8%2.5%
Best scripted twinPython, at most two plies89.8%51.0%
Best written block*One new block, trained interfaces21.3%1.8%
Centre-column ruleNearest legal column to the centre4.5%0.5%

The written-block row counts only under the second reading. It replaces the original eight-block body with one written block, so it is not another repair of the full model. The trained self-play score need not land exactly at half under the finite, noisy game protocol.

Dot chart of game scores, as a share of points, against a depth-4 search bot and head-to-head against the trained network. Full models: trained 93.0% and 51.3% in self-play; training recovery 88.5%, head-to-head not measured here; convert-only 12.8% and 2.5%; submitted repair 11.5% and 1.5%; the two unchanged repair attempts 12.8% and 2.5%. Best written block, second reading only: 21.3% and 1.8%. Best scripted twin 89.8% and 51.0%; centre-column rule 4.5% and 0.5%. Below, all six W2 attempts in order: writers void under the primary reading and fail, partial, fail under the second; twins pass under both, head-to-head 43.0, 51.0 (interval 44.1 to 57.8) and 22.5.

Scores are shares of points in 200 games per cell (a draw counts half), with colours alternating and 5% random moves for both players. The written block counts only under the second reading and replaces the whole body with one block, so it is drawn apart from the full-model repairs. From the frozen evaluation records.

The practice interface helps explain the contrast, but does not isolate its cause. Both arms had 50 practice calls and free self-tests on positions they supplied themselves. The writers used only three to six practice calls. The twins could also run and inspect their own Python logic freely, including their self-written solvers. Weight writers faced the guard on chaining network operations outside the designated tools. This comparison changes the representation, architecture and ease of debugging together. It shows which route worked better here, not which single restriction caused the gap.

Claude Code estimated plan usage at about 62forthereadtest,62 for the read test, 27 for the sanity pair, 58.5forrepairand58.5 for repair and 189.5 for writing and its twins. These are usage estimates, not a total project cost or a billing comparison with training.10

Conclusion: two checks worth taking

We tested whether one frontier model could read, repair and write the weights of a small ternary game player. It recovered genuine structure, including an exact hidden-unit shuffle. Its diagnoses nevertheless followed layout and architectural assumptions. Given the conversion history and practice feedback, it did not recover playing strength within the repair budget.

Writing was more encouraging in a narrow way: under the second reading, one constructed block scored higher than the weak converted file against all three search bots. It did not improve the head-to-head score against the trained network, and its value preservation was roughly level with the converted file. But the same model did much better writing a small Python player. That comparison gives someone considering weight-space programming a practical starting point.

Use a function-preserving shuffle to ask whether a diagnosis follows behaviour or appearance. Use a scripted twin to ask whether writing weights buys anything over writing the program. Both controls made this experiment more informative than a demonstration on its own.

These results concern one language model, one architecture and one game under the rules described here.11 A follow-up article will report the same tests with a model running on our own hardware.

The appealing idea remains: perhaps weights can become something we deliberately read and edit. Here, finding a pattern was easier than knowing what it meant. The game was a useful way to tell the difference.

Download the evidence package (ZIP, r01) for the result records, task texts, scoring scripts and selected answers. Its README maps the claims to records; paths in the footnotes are relative to the archive's weight-space-programming-evidence/ directory.

Scripts are MIT-licensed and text/data are CC BY 4.0; see LICENSE-NOTE.md. Model checkpoints are available separately.2 Submitted written blocks and twin programs, full tool logs and the evaluation harness are not included; the archive records submission and harness hashes.

Footnotes

  1. Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, MIT CSAIL, Sparks of Weight-Space Programming. Their post motivates the read/edit/write questions. Our ternary game experiment uses a different model and protocol; it is not a replication of their reported tasks. ↩

  2. Precisit, A game-playing AI in 1.6 MB, and the OnePass Connect Four model card and files. T34 is the ternary packing used here. Hugging Face hosts the trained file; the trained checkpoint and convert-only checkpoint are both available at this pinned public GitHub commit. They share an architecture but have different training histories. ↩ ↩2

  3. Evidence package (ZIP, r01): docs/wsp/W1-RESULT.md and results/wsp/w1-score.json, fields attempts[].kind, pick, right, counted, confidence, guard_violations and kinds. The shuffle check is in results/wsp/w1-permutation-parity.json, fields parity.positions, parity.same_choice and parity.bit_identical_rows. The two displayed layer-agreement quotes are in results/wsp/w1-runs/w1-63c24fca/answer.json, field mechanism; that attempt has counted: false in the scorecard. Agreement ranges are computed from the separately linked checkpoints and the shuffle described in the archive README. ↩ ↩2 ↩3

  4. Evidence package (ZIP, r01): results/wsp/w1-runs/w1-048667b1/answer.json, field mechanism, contains “planted rank-1 components”. This is the reader's rejected diagnosis, not our description of a defect. Its entry in results/wsp/w1-score.json, attempts[], has counted: true. ↩

  5. Evidence package (ZIP, r01): docs/wsp/W1-RESULT.md, “Sanity pair” section, records the six reported observations and their hygiene exclusions. The rotary quotation is in results/wsp/w1-runs/w1-a411617d/answer.json, field mechanism. All six sanity attempts, including the quoted one, are void and are reported rather than counted; none attempted a forward pass. The architecture uses learned absolute positions, and the answer schema forced a choice. ↩ ↩2

  6. Evidence package (ZIP, r01): results/wsp/w3-score.json, fields attempts[].edited_trits, attempts[].eval, start and qat_comparison; docs/wsp/W3-RESULT.md describes the three attempts and the earlier training recovery. tasks/w3-repair-task.md specifies the 1% edit budget and 50 practice calls. “distributed quantization noise” is in results/wsp/w3-runs/w3-084de542/answer.json, field approach; the scorecard marks that attempt as counted. ↩

  7. Evidence package (ZIP, r01): docs/wsp/W2-RESULT.md and results/wsp/w2-score.json / results/wsp/w2-score-second-reading.json, fields reading, cards[].counted, verdict, value_kept, vs_bot1, vs_bot4, vs_bot6, head_to_head_vs_trained, passes, selftests and baselines. The best writer is w2-fb31b1d0; the best twin is w2-aede43e3. In results/wsp/w2-eval/w2-aede43e3.jsonl, the kind: head_to_head row supplies candidate_score and a_score_ci: the candidate interval reverses and subtracts the trained side's bounds from one, giving 0.441 to 0.578 around 0.510. This is a roughly level result, not an established advantage. ↩ ↩2

  8. Evidence package (ZIP, r01): docs/wsp/W2-PROTOCOL.md, amendment 10, and results/wsp/w2-review.json, fields before_any_eval and decisions[] (attempt, cell, void, reason). The review covers all 110 flagged steps. Both w2-score.json and w2-score-second-reading.json retain their respective reading, writer_verdicts and twin_verdicts. The original guard reading remains primary; the alternative reading was chosen before final scores were available. ↩

  9. Evidence package (ZIP, r01): docs/wsp/EVAL-FROZEN.md, “What it runs” and “Frozen hashes” sections, defines the held-out positions, game conditions and harness identity. In the results/wsp/w2-eval/ and results/wsp/w3-eval/ JSONL records, kind: positions rows report n, value_preserving and phases; game rows report games, noise, seed, draws, a_score and a_score_ci. A draw counts half a point. The harness is identified by hash but is not included in the archive. ↩

  10. Evidence package (ZIP, r01): docs/wsp/W1-RESULT.md (main test and sanity-pair sections), docs/wsp/W3-RESULT.md and docs/wsp/W2-RESULT.md report the Claude Code plan-usage estimates. These are not the full cost of engineering, evaluation or training. ↩

  11. We have not yet run the proposed whole-body writing test or the separate Base243 comparison. They are outside the scope of this article. ↩

Keep readingWhat could a megakernel win on a Mac?Four games, four small models ← All articles
100%