A Swedish form specialist in one forward pass

An email address is already in the document. An empty field on a form asks for “E-postadress”. Somewhere between those two things, software has to decide that they belong together. Repeated across a form, those small decisions become work somebody has to do.

A large language model can help read the document and understand the task. But once the application has the values and knows which actions are available, could a much smaller model handle the repeated choices?

That is an appealing role for a local specialist. Each decision could happen inside the application, without a network request or a generated answer to interpret. Ordinary code would still control the workflow and check the result. The aim is to make routine work easier to automate while keeping the model's job small enough to inspect.

We trained a Swedish form specialist with 706,048 parameters, using an open architecture and synthetic data. It chooses an option in one forward pass. The released checkpoint scores 99.29% on held-out synthetic decisions. On two simplified test fixtures based on forms from our own websites, it scores 52.8% and 77.3%.¹²

The practical question is how far that small component can be trusted with the job. To answer it, we looked beyond the headline: at the fields it leaves empty, forms outside its training generator, and what happens when the same model runs through a different execution path.

Give the model a menu

In September, TypeSafe introduced Jev and its System One interface: supply a state, ask structured questions, and receive typed decisions and probabilities. The attraction is a short path from a model's answer to ordinary application code.³

Open projects explore related ideas. jevlike implements an option-scoring attention head. Cua-S1 applies that approach to form decisions, with a tiny byte-level model and synthetic training data. Laya explores a broader set of typed decisions with pretrained encoders, including multilingual models. These are different implementations and training recipes, not interchangeable versions of Jev.⁴

Our Swedish experiment uses Cua's model and trainer, with the upstream code vendored in the toolkit and credited to its authors. We trained new weights from scratch. The Swedish catalogue, generator adaptation, evaluation and Core ML tooling are the work described here.⁵

For one empty field, the interface looks roughly like this:

Context:
  Form: patient registration
  Element: text field, label “E-postadress”, current value empty

Available options:
  fill First name: Anna
  fill Email: anna@example.invalid
  check
  click
  skip

The actual inputs use Swedish labels and action words. The example is translated to make the contract easier to see: the candidate value is supplied to the model. Reading a document and extracting that email address is a separate job.

The model encodes the context and options, uses each option to attend to the context, and produces a score for each choice. All candidates for that element are scored together. There is no token-by-token answer to generate or parse.

This is one pass per element's decision, not one pass to complete an entire form. Code still decides which element to inspect next, validates the chosen action, enters the value and checks what happened. A model restricted to a menu cannot invent a new email address, but it can choose the wrong supplied address, skip a useful field or click at the wrong time.

For a form we own and understand completely, explicit field mappings may be the better solution. The interesting case is variation: labels, synonyms and partially completed forms that make those mappings less straightforward.

Teaching the vocabulary

The Swedish catalogue contains 50 concepts. Each connects labels used on a form with labels that might accompany a value in a document, plus a generator for synthetic values. For email, the form labels include “E-post”, “E-postadress” and “Mejladress”; document labels include “E-post”, “E-postadress” and “Mejl”.⁶

The generator combines those concepts into form episodes. It varies which entities are present, adds distractors and creates partially filled states. A single episode produces several labelled decisions: fill this field, leave that one alone, check a box, or click the button. The split is assigned from a form signature derived from its field concepts. Episodes with the same signature, and all their decisions, stay together in training, validation or test.

The split does not hold out an entirely different world of forms. All three parts still come from the same catalogue and generator. That matters when we interpret the score.

We ran three corpus sizes with the corrected generator, keeping the model, four-epoch schedule and batch size of 64 unchanged.⁷

More generated episodes, with the same four-epoch recipe. Each run has its own test split.

The largest corpus contains 234,921 training decisions, 29,649 validation decisions and 29,839 test decisions. Its four training epochs took about 163 minutes on an Apple M4 machine. The corpus seed is 2026 and the training seed is 7; the learning rate is 0.002.⁷⁸⁹

There is a large improvement between the first two runs and a smaller one between the last two. We should resist turning those three points into a law. Larger corpora also mean more optimisation steps at a fixed epoch count, and each run has a different test split. This experiment does not separate those effects or establish that training has reached a plateau.

For comparison, the unchanged English Cua checkpoint scores 20.75% on the same 29,839 Swedish test decisions. That is a useful warning about transferring this particular specialist to new inputs. It is not a general measure of English versus Swedish capability: the labels, action strings and generated data distribution all change.¹⁰

What the score leaves out

Top-1 accuracy asks whether the highest-scoring option is exactly the expected one. It is a useful first measurement, but different mistakes have different consequences.

On the largest synthetic test, almost half the expected actions are “skip”. A system that always skips would score 47.36% without filling anything. To understand 99.29%, we also need the action breakdown.

Of 13,844 expected fill decisions, the released model silently skips one. Here, “silent skip” means choosing “skip” when the test expects a fill. It does not include choosing the wrong value, which is another way a fill decision can fail. Fill accuracy is 99.00%, slightly below the overall score.¹

We also deliberately mismatch contexts and option lists. In the synthetic evaluator, contexts are rotated between examples within a batch while options and expected answers stay put. Accuracy falls to 31.99%. The model is using the context; the score does not come solely from regularities in the options. That control still cannot tell us which contextual clues it has learned.

A small handwritten set supplies another check. The model gets all 50 decisions right, but those are 32 fills, 12 skips, three checks and three clicks. That is a useful set of examples to inspect, not enough evidence for a broad claim about Swedish forms.

Beyond the generator

We then built fixtures based on the interest forms for Kanslist and Pratsam. They combine field definitions with synthetic values and a hand-written table of expected actions. Fifteen scenarios per form produce 195 Kanslist decisions and 75 Pratsam decisions.²

These are simplified representations of the forms, not live browser tests or real submissions. Some labels and controls differ from the current website implementations, and the same fields recur in all fifteen scenarios. The numbers measure decisions on these fixtures, not the percentage of visitors whose forms would be completed successfully.

The Kanslist result is especially revealing. The model leaves 57 of 65 expected fills undone. These include optional website fields for which a synthetic scenario supplies a value; they are not all required fields. A harmless-looking “skip” can still mean that the task was not done.

Pratsam's higher aggregate score needs care too. There are only eight expected fills, all email addresses. The model gets those right, but gets none of the expected checks or clicks right. Many of the successful decisions are skips.

Reviewing the fixture code exposed a problem in the test itself: the radio-choice scenario uses “mig själv” (“myself”), while its expected-action logic checks for “ja” (“yes”). It therefore labels every radio decision as “skip”. That makes this part of the Pratsam score provisional. Reproducing a number is not the same as validating what it measures.

There are several things to investigate next. Kanslist asks about an association's role, apartment count, property manager and communication tools. Much of that vocabulary lies outside the training catalogue. But vocabulary is a hypothesis, not a complete explanation: control roles, input lengths and the fixture's own mapping conventions also differ.

Before retraining, the useful next step is to align these fixtures with the actual controls and repair the expected answers. Then we can hold out a reviewed test set and vary the training vocabulary. More episodes from the existing generator would not, by themselves, test whether those missing concepts have been learned.

Small enough to keep nearby

One attraction of a model this size is that it can stay close to the application. We exported the released checkpoint to Core ML and measured three package variants. A follow-up comparison on an Apple M4 tested every variant with the same inputs under two execution configurations.¹¹

All 29,839 held-out decisions, with no warm-up rows excluded from accuracy. “CPU + Neural Engine allowed” means Core ML's CPU_AND_NE setting; it does not establish where each operation actually ran. A KiB is 1,024 bytes.

The 787 KiB package remains a useful result. The 8-bit recipe compresses weights using a 256-entry palette while computation remains float16. With CPU and Neural Engine allowed, it retains the same rounded accuracy as the float16 package, although 13 decisions differ from the PyTorch baseline. “Same score” is not “same answers”.¹²

In the earlier timing run on the M4, that configuration's median prediction time was 1.314 ms, with 1.411 ms at the 95th percentile. Those measurements cover MLModel.predict over 29,836 decisions after three warm-up examples, excluding input encoding and model loading. They are not end-to-end form timings. For context, the float16 package's CPU-only median was 1.727 ms on the same setup.¹³

The smaller package changed our interpretation of the experiment. We had first scored it with CPU and Neural Engine enabled, where accuracy fell to 49.92%. It looked as though four-bit weights had lost too much information. But the same package reached 99.27% on the CPU, with only 32 changed choices against PyTorch. The rounded weights still supported the task. The much larger failure depended on how that export ran.

We checked the released packages on another machine, an M1 Max, using 512 decisions sampled across the test set. Int4 got 511 correct on CPU and 266 with CPU and Neural Engine enabled. Expanding its compressed weights into ordinary float constants, keeping the already rounded values, produced exactly the same choices under each setting. That did not repair the failure.¹⁴

The exact cause remains open. The run emitted ANECCompile() FAILED messages, but neither those messages nor the predictions tell us which operations ran where. There is also an export difference to investigate: float16 and int8 target iOS 17, while int4 targets iOS 18. Matching those targets and inspecting the resulting graphs is a more useful next step than retraining to compensate for an unexplained runtime difference.

There is another constraint behind the small package: the exported model accepts 224 UTF-8 bytes of context, 96 bytes per option, and at most 40 options. Those are bytes, not words or characters. Long labels and multibyte characters consume the budget, and overlong text is truncated. The toolkit rejects an option list above the fixed ceiling instead of silently dropping choices.¹⁵

Size, latency and accuracy belong to a particular input contract and execution configuration. Testing the checkpoint alone does not test that whole arrangement.

Inspect the instruments too

The work also found two bugs in our synthetic identifier generators. Their value shapes were wrong. Different shapes can give a model an accidental shortcut: recognise the punctuation and digit pattern instead of the field's meaning. Tests now check the intended shapes and checksums, and the released checkpoint uses the corrected generator.⁶

An early retrain scored lower after that correction. It was tempting to explain the drop entirely as the removal of a shortcut. The later, larger runs made that story less convincing. Neither comparison isolates the cause: the generated examples and test sets changed, and the larger runs received more training updates. A shape audit identifies a possible shortcut; it does not measure how much the model relied on it.

Export needs its own instruments. The toolkit checks an export-friendly PyTorch implementation against its traced graph on 512 examples before conversion. It then measures Core ML decisions against PyTorch on the evaluation data. Those checks answer different questions: whether tracing preserves the implementation, and how much the deployed package changes its answers.¹²

Taken together, the useful checks are quite ordinary: inspect the generated data, count mistakes by action, mismatch the contexts, compare answers after export, and try inputs that did not come from the generator. The fixture bug adds one more: read the code that decides what “correct” means.

Try the released checkpoint

The toolkit, model and Core ML packages, and synthetic corpus are public. The corpus manifest records the split sizes and file hashes.

To inspect the two form fixtures without repeating the training run, use this snapshot. The commands below were checked with Python 3.12 and PyTorch 2.7.0 on macOS; the fixture evaluator runs on the CPU.

git clone https://github.com/precisit/one-pass-specialists
cd one-pass-specialists
git checkout 6a6def7887ea13a323049a972409b19aed953559

uv venv --python 3.12
uv pip install --python .venv/bin/python -e '.[train,dev]' 'torch==2.7.0' huggingface-hub

.venv/bin/hf download precisit/one-pass-sv-forms sv0-forms.safetensors sv0-forms.json --revision 778ff920104b36800a9f028521ad958eedcb8b7a --local-dir runs/released
.venv/bin/python -m pytest -q
OMP_NUM_THREADS=2 .venv/bin/python examples/sv-forms/build_real_forms.py --checkpoint runs/released/sv0-forms.safetensors --out fixture-results.json

All 13 toolkit tests pass at this revision. The fixture run reproduces 52.8% and 77.3%, including the limitations described above. It writes the decision rows and fixture-results.json, so you can inspect a field, its candidates and its expected answer together. Correcting a fixture will change the experiment; keep the old and new results separate.

Conclusion: a small model needs a well-defined job

We began with a practical question: could a small local model take care of routine form decisions while ordinary code kept control of the workflow? The size and timing measurements make that role plausible. A 787 KiB package can score one element's options in about 1.3 ms on the measured M4 setup, and the Swedish training run shows how much a compact model can learn from a bounded catalogue.

The checkpoint is not ready to fill unfamiliar forms unattended. The 99.29% synthetic score does not cover the missed fills and weak action choices in our website fixtures. Those fixtures also need their own corrections before they can guide further training. The next step is to repair and review the expected answers, then test whether a broader catalogue helps with held-out examples of the actual task.

The Core ML result adds a second boundary. Four-bit weights preserved 99.27% accuracy on CPU, yet the same package gave very different answers with another execution configuration. A useful local specialist therefore needs evidence about both the decisions it has learned and the way the application will run it.

For an application with fixed, known fields, explicit mappings may still be the best tool. Where the labels and states vary, this is a small component worth testing: supply the allowed choices, measure the consequential mistakes, and keep execution and verification in code. The published catalogue and evaluation tools give another team a place to start that experiment with its own task.

The appeal is the ordinary work it might take off someone's hands. A tiny model does not have to understand everything to be useful. It does need to get its part of the job right.

Footnotes

1. Released PyTorch evaluation results, including action counts, handwritten examples and the context control. The 99.29% headline rounds the reported 0.9929 top-1 score. ↩ ↩²

2. Public fixture builder and expected-action logic and fixture results. The context-shuffled controls score 42.1% for Kanslist and 64.0% for Pratsam. These fixtures need review before being treated as faithful tests of the shipped controls. ↩ ↩²

3. TypeSafe's introduction of System One models and Jev, 15 September 2026. We make no comparison with Jev's accuracy or latency here. ↩

4. Primary projects: jevlike, Cua-S1 source, Cua's form checkpoint, and Laya. See each project's own scope and licence. ↩

5. Toolkit and third-party attribution. Cua source is vendored from 9bbfa7d; the credited jevlike attention-head design comes from 94f5fd1. ↩

6. Swedish concept catalogue, generator and catalogue tests. Earlier generator behaviour is described in the model repository's release notes. ↩ ↩²

7. Published three-run size sweep and run summary. Timings describe the original training runs; we did not repeat training for this article. ↩ ↩²

8. Published synthetic corpus manifest. The final corpus has 10,000 generated episodes, with episode-level train, validation and test assignments. ↩

9. Released checkpoint configuration and training metadata. Deterministic training was disabled; seeds alone do not promise an identical rerun. ↩

10. English checkpoint evaluated on the Swedish rows. The raw score is 0.20748. The unchanged English checkpoint receives the Swedish action strings as well as Swedish field labels. ↩

11. Full-set CPU/CPU+NE comparison and measurement script. Reported platform: M4, macOS 26.6, coremltools 9.0. All 29,839 rows are included. Core ML's compute-unit setting permits a set of processors; it is not a placement report. ↩

12. Released conversion metadata. The 8-bit recipe is uniform per-tensor palettization; 4-bit uses grouped-channel K-means with groups of 32. Both use float16 computation. The trace check compares the export-friendly eager implementation and its trace, not a proof of bit-identical equivalence to the original forward implementation. ↩ ↩²

13. Released Core ML evaluation and timing results and evaluation implementation. Reported sizes sum package file bytes, not resident runtime memory. Original latency measurements used an Apple M4 machine; the M1 Max follow-up measures accuracy only. ↩

14. M1 Max verification results and reproduction script, including artifact hashes, sample indices and individual predictions. macOS 26.5.1, coremltools 9.0, PyTorch 2.7.0. The deterministic 512-row sample is a diagnostic check, not a replacement for full-set accuracy. Apple's weight decompression utility removes the 17 palette-expansion operations; this also runs conversion passes, so it is not proof of an otherwise identical graph. The decompressed package retains the rounded weights and the iOS 18 target. ↩

15. The shared byte encoder and input limits. UTF-8 bytes are shifted by one so zero remains padding. Text truncation and the fixed option ceiling are part of the export contract. ↩