Four games, four small models

An opponent in a browser has to arrive before it can play. For a game that someone opens during a pause, the download is part of the experience. So is being able to return to it without a connection.

Woodpip offers Four in a Row, Five Dice, Reversi and Backgammon against a local opponent. Each game has a small neural network that scores its legal moves. Three of the model files are about 1.6 MB; Reversi's is 4.7 MB. Those are model downloads, separate from the site's code and runtime memory. The site's pages and engine are kept after the first visit, and each game's opponent once that game has been played, so it then plays offline.

The models share a recipe: learn from a teacher, compute with ternary weights from the first training step, and check the exported file through the browser's actual move path. Making that recipe useful across four games also exposed a boundary. Reversi benefited from an exact solver that takes over near the end.

The companion article, Designing Woodpip for a few quiet minutes, follows the product decisions around these models. Here we follow the training, runtime checks and measured play.

OnePass: learn to choose a move

The input is deliberately narrow: a position as fixed-length text and each legal move as a short text. The transformer produces a score for each move. The game supplies the legal choices; the model scores them. This is the OnePass approach from our Connect Four work, where we explored the System One idea behind models such as TypeSafe's Jev: a learned decision over supplied choices, without generating an answer token by token.¹

That is useful when the surrounding program already knows what a valid action looks like. The rules engine can enumerate the legal moves, while the model learns which ones are worth taking. Evaluating the network uses what it learned during training without searching a tree of future positions. This gives us a small, local component with a result the game can use directly. It still has to earn its playing strength in tests.

The public OnePass toolkit documents our independent implementation and its open-source lineage.² “One pass” describes the model's way of scoring choices. Backgammon can split a long list of plays into several batches, and Reversi can hand a late position to an exact search. Both are useful parts of the opponent described below.

A teacher grades the choices

Training starts with a teacher that can value those choices. Four in a Row uses an exact solver's win, draw or loss values. Five Dice uses yatzy-solver, whose dynamic programming gives the expected remaining score of every action. GNU Backgammon supplies equities, its values for the legal plays. Reversi's teacher, Edax, supplies exact endgame values and fixed-depth search values earlier in the game.³

One model file per game, with counts and sizes rounded to two decimal places; MB means one million bytes. Reversi has the larger model; its exact endgame solver is a separate part of the opponent.

The teacher's values become a target distribution over the legal moves through a softmax. This turns the values into probabilities that sum to one. The model's own scores pass through a softmax too, and cross-entropy measures how far its distribution is from the teacher's. Training adjusts the model to reduce that difference. The teachers work offline; their program code is not shipped as Woodpip's opponent.⁴

Five Dice's target rule applied to one decision. The values and the model's probabilities are illustrative; the arithmetic is the training loss.

The positions include both strong play and deliberate randomness. Five Dice's six million training rows mix optimal play, play with random decisions, entirely random play and uniformly sampled situations. Backgammon's 10.5 million positions come from games mixing teacher-guided choices with random moves. Reversi uses five rounds; from the third round, the model's own games supply positions for the teacher to label. The final round alone contains 7.57 million positions. This last process is dataset aggregation, or DAgger: the teacher helps with positions the pupil actually reaches.⁵

Reversi's five rounds. The loop under rounds 3 to 5 is dataset aggregation.

Splits are by game or seed, with validation used for model selection and a separate evaluation set reported once per model. Protocols and selection gates were written before the first training runs. Failed models remain in the record too.

Four teachers, one recipe: each game's model learns from its own teacher. The split bar shows the roles of the three sets, not their sizes.

Learn with the small representation

All four models were trained with Apple's MLX on Apple silicon.⁶ The common trainer uses Adam, a warmup followed by cosine learning-rate decay, and gradient clipping. Four in a Row follows the published recipe from our ternary-model article; the other games use the same training approach with their own data and schedules.

T34 applies ternary encoding to the encoder matrices. In each group of four weights, the smallest magnitude becomes zero and the other three keep their signs. A scale shared across 128 weights supplies the magnitude, so each weight becomes minus the scale, zero or plus the scale. The 3:4 encoding comes from Sherry's ternary format.⁷ The forward pass uses this representation from the first step, so the model learns while computing with the restriction that will shape its download.

The rest of the model uses other representations: the embedding and the head's query, key and value projections use int8, while norms, biases and other small parts use fp16. Their quantization is also simulated during training. The whole model is therefore a mix of formats, as the export diagram shows.

The optimizer still keeps floating-point latent weights. A straight-through gradient allows training updates to pass through the quantization step.⁸ “Ternary” describes the weights used for those computations and the shipped representation; it does not mean that the optimizer's entire state contains only three possible numbers.

The result is an Open Neural Network Exchange (ONNX) file containing packed weights and a graph. Its ternary matrix products use custom nodes, with a reference implementation inside the model that decodes them using standard operators. ONNX Runtime can therefore run the same exported file unchanged. The browser can be checked against that reference without making a second model file just for the comparison.⁹

Training keeps floating-point latent weights. Encoder matrices compute with, and ship in, T34; the other parts use int8 or fp16. Only the exported file reaches the browser.

Five Dice gives us a view of this training in one uninterrupted run. In the validation plot, regret is the expected score given up by choosing the model's move instead of the teacher's best, measured per decision. That gap narrows over the recorded checks, while agreement with the teacher's best moves rises. These checks select a checkpoint; the separate game evaluations below establish how it plays.¹⁰

The shipped Five Dice model's training run, as recorded. The training loss includes the targets' own entropy, so only its shape is meaningful. Validation, a fixed 40,000-row subset, only selected the checkpoint; the published result, 0.027 points per game, comes from the separate evaluation set and is not on these curves. Validation loss was not logged.

Reversi needed a second way to choose

Two pre-registered experiments found no gain from deeper teacher labels (search depth 20 instead of 12) or from a model larger than the shipped one. An exact endgame solver offered a useful complement: it could resolve late positions that the model had learned to estimate.

At Expert, the engine hands over when 14 or fewer squares are empty. At Challenging, the threshold is 10. Relaxed does not use the solver. It runs in a Web Worker, keeping the search off the page's main thread. If a solve has not finished after two seconds, the model chooses instead. Exact play is conditional on the solver finishing; empty squares are not a promise about the number of moves left.

In a 400-game head-to-head experiment, model plus solver scored 0.581 against the model alone, with a 95% interval of [0.532, 0.629]. A win counts one and a draw half. This combined opponent is search-assisted. The experiment is separate from the later measurements of the shipped difficulty levels below.

Two games at Expert, before and after the handover. In A, with 38 empty squares, the model chose Woodpip's marked move; in B, with 13, the exact solver chose g8.

Who chooses each Reversi move. Dashed lines mark the two games shown above.

Check the opponent that reaches the browser

A good evaluation of the trained model leaves another question: is that what the player actually runs?

Every model load checks the file's size and SHA-256 hash. Rule tests compare reference positions byte for byte. Fixed positions then pass through the real move path and are compared with recorded ONNX Runtime scores. The largest score difference must be below $10^{-3}$, with the same best move. Reversi also tests its solver.

These checks answer different questions. A hash identifies the file. A score tolerance checks numerical agreement. A move comparison checks the decision. Agreement with a reference runtime does not, by itself, establish that the decision is strong.

On the complete evaluation sets in headless Chrome, using the real GPU of an Apple M1 Max, the browser's selections agreed with ONNX Runtime on the CPU as follows:

The WebAssembly CPU fallback reproduces the same counts on the same model files. Reversi's solver is switched off for this comparison.

The one Five Dice difference was a tie: the two scores were $5\times10^{-7}$ apart, and the teacher valued both moves as best. Reversi's solver received a separate check on 8,372 positions with at most 14 empty squares. Each chosen move matched the teacher's best value, and every move's exact value matched its label. Four further positions were excluded because their teacher labels were invalid.

The engine prefers WebGPU. If it is unavailable, or a model fails its GPU self-test, the same file runs through WebAssembly SIMD on the CPU. SIMD means single instruction, multiple data: an operation can work on several values together. Our runtime article describes the earlier browser work behind this path.

Backgammon adds a useful implementation detail. A position can have hundreds of legal plays, so the runtime scores them in chunks of 64. Each play's score is independent of the other plays' scores, which makes that chunking possible. A model that scores choices without searching future positions can still require several runtime calls to score a turn.

Measure what the levels mean

Expert chooses the model's best move, with Reversi's solver handover as described above. Relaxed and Challenging sample from the model's move probabilities using a fixed temperature, plus a small chance of choosing a uniformly random legal move. The settings were calibrated per game and confirmed with a fresh seed. They do not adapt to the player.

The results need their opponents and units attached. Four in a Row and Reversi report a game score: wins plus half the draws, divided by the number of games. Both sides make 5% random moves in these evaluations. Reversi's games alternate colours from 100 fixed openings. Backgammon uses duplicate dice: each pair plays the same dice with sides swapped. Five Dice reports solo final scores.¹¹

Four in a Row: 200 games per level.

Five Dice: 20,000 solo games per level.

Backgammon: 2,000 games per level against GNU Backgammon at 0-ply, forming 1,000 duplicate-dice pairs.

Reversi: 200 games per level.

These tables measure the shipped level settings. Reversi's Expert result is a separate measurement from the model-plus-solver experiment above.

The same twelve results, each game on its own scale. Where opponents change between levels, a lower row is not a weaker setting.

Five Dice lets us separate decision quality from the luck of the rolls. Perfect play under its Scandinavian Yatzy rules averages 248.44 points. The solver can also value each decision the model takes. Across 20,000 games, the shipped model gave up an average of 0.027 points per game, with a 95% interval of [0.026, 0.029]. That corresponds to a reported expected score of 248.41.

The sampled mean of 248.1 in the table includes dice noise. Its interval, [247.6, 248.7], includes the perfect-play average. That does not establish perfect decision-making; the solver's decision-loss measure exposes the small gap. On the site, you and Woodpip roll separate dice and compare totals. These solo-play measures are not head-to-head win rates.

A shared link has a different job

The opponent runs locally. A chat app fetching a shared link still needs server-generated metadata to display the right picture.

Woodpip puts the game's moves, level and, for dice games, its seed in the link. A small responder supplies the replay page's Open Graph tags. A card function replays the moves through the same rules used by the site, draws the final position with the site's drawing code, and rasterizes it with resvg and bundled fonts. It does not need a headless browser.¹²

Replay is also validation. An illegal or malformed link gets the generic game card. The picture contains a legal position and fixed copy; a link cannot supply arbitrary words for the card. The released card images are checked against the images approved during design.

No game is saved on the server for this purpose. The picture is cached at the edge, and an evicted picture can be drawn again. If the responder fails, CloudFront falls back to the static site.

That architecture has a precise privacy boundary. The replay URL's query string contains moves, but access logs omit that query string. The replay page sends no referrer. The card image's path does carry the moves, and that request is logged for 90 days. Standard server access logs also include IP addresses. The card functions log the game, outcome and time taken. Local play and the absence of tracking scripts do not make those server records disappear.

The path of a shared link. The saved game stays in the player's browser; the link carries its moves.

From a learned decision to a playable opponent

So what did we do? We trained small models to score legal moves, using solvers and search programs to teach them how to value the choices. We trained with the compact representations they would ship in, exported each model to one file, and checked the browser's decisions against the reference runtime. The resulting opponents can run on the player's device and remain available after their first download.

We also measured what those decisions amounted to in play. A lower training loss, agreement between runtimes and a stronger opponent answer different questions. Five Dice's solver separates mistakes from dice luck; the level tests put results beside named opponents and conditions. Reversi made the limit of a model-only design particularly useful to see: neither deeper labels nor a larger model improved the reported results, while an exact solver improved late play when it could finish the search.

The useful unit of work was the whole path: a teacher's values, a training constraint, an exported file, a runtime check and a measured game. Each step answered a question that the previous step left open. Reversi also showed where a deterministic solver could improve the opponent.

For further study, the public pieces include the OnePass model toolkit, ternary training and formats, the WebGPU runtime, and yatzy-solver. The Four in a Row model is published as onepass-c4-v3-t34.onnx.

The four games put those decisions in a place where they can be used. Open Woodpip, choose a game and take a turn.

Footnotes

1. TypeSafe introduces its decision-model interface in Introducing System One models and Jev. Our earlier OnePass article explains the connection and applies the idea to Connect Four. ↩

2. The public OnePass toolkit documents its independent implementation. Its third-party notices credit the open Cua-S1 and jevlike lineage and distinguish it from TypeSafe's Jev code, weights and API. ↩

3. The teachers' own projects document their methods: yatzy-solver, GNU Backgammon and Edax. The exact Connect Four teacher is described in our first model article. ↩

4. For the teacher/student idea and learning from soft targets, see Hinton, Vinyals and Dean, Distilling the Knowledge in a Neural Network (2015). Here the teachers are game solvers and search programs, and the student learns their distribution over legal moves. ↩

5. Ross, Gordon and Bagnell, A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning (2011), introduces DAgger: collecting teacher labels on states reached by the learner and adding those examples to training. ↩

6. MLX documentation describes the array and neural-network framework for Apple silicon. Our public ternary-training project contains the Four in a Row training and export work behind the recipe. ↩

7. Huang et al., Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-Grained Sparsification (2026). The 3:4 codes pack four weights into five bits; scales and the model's other formats add storage beyond that. Our format implementation documents the encoding used here. Using the format does not mean reproducing every part of Sherry's training method. ↩

8. For the straight-through gradient estimator, see Bengio, Léonard and Courville, Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation (2013). It provides a way to pass an approximate gradient through a non-differentiable operation such as quantization. ↩

9. The ONNX intermediate-representation specification describes model-local functions. They can supply a standard-operator body for a custom operation, while a runtime may provide an optimized implementation. This is what lets the packed model carry its own reference decoding. ↩

10. The recorded training and validation data (JSON) contains the surviving histories of all four shipped models, with metric definitions, stage boundaries, selected checkpoints and model-file SHA-256 identities. The Five Dice figure uses the logged batch losses and their exponential moving average (0.98 per step), without further smoothing. Loss levels are not comparable across games: their teacher targets have different entropy, and their training mixtures differ. Validation loss was not recorded; the validation plots show decision quality, not loss on a held-out batch. ↩

11. The game-score intervals for Four in a Row and Reversi use Wilson score intervals with draws counted as half. Five Dice's mean final score and decision-loss intervals use the mean plus or minus 1.96 standard errors over games. Backgammon uses the same mean-based construction over duplicate-dice pairs, with each pair counted as one sample. All intervals shown are 95% intervals from the reported measurements. ↩

12. resvg-js provides Node.js bindings to the Rust resvg renderer, turning SVG drawings into raster images. ↩