Research Note · lesion X-ray · matched pair

What 1,800× Less Data Costs You — and What It Doesn't

Tetracta AI Teams · published 23 August 2026 · all numbers from our own harness

What this note does not claim. Attributing an internal difference between two checkpoints to architecture requires independent-seed controls, and we have not run them here. In July 2026 we published internal differences between two arms and retracted them after three purpose-trained control models showed the figures sat inside ordinary run-to-run noise. The gaps in this note are 8–20×, not noise-band — that makes them more interesting, not more controlled. Read every internal number below as "these checkpoints differ by X under this instrument".

The short version

We benchmarked our 1B constant-state model against a modern open transformer that saw roughly 1,800× more training data. We lost by 20.3 macro points. That is the honest headline and we are not going to dress it up.

Then we ran a different instrument — a lesion X-ray that damages one layer at a time and measures how far the output distribution moves. On that axis the ordering changes. Against our own transformer twin, trained on identical data, our model is 7.9× more robust before fine-tuning and 9.1× after, and its worst single layer is 20.5× less critical.

Two instruments, two different answers, and they are not in conflict. Benchmarks ask what does it know. The X-ray asks how is it built.

1 · The benchmark, first, because it is the bad news

Ten standard tasks, our frozen harness, same seed, same 1,024-token cap, same metric contract for every model. Nothing was re-run after seeing a result.

Macro benchmark scores for nine 1B-class models measured on one frozen harness; Qwen2.5-1.5B-Instruct leads at 66.36, Z-Next v4.3 sits at 45.52, ahead of its twin and both Pythia checkpoints.

We are 20.3 points behind Qwen. We are also first in our own budget class — ahead of our twin, ahead of both Pythia checkpoints trained on comparable token counts. Neither sentence cancels the other; both go in the table.

2 · The X-ray

The method is deliberately crude, which is what makes it hard to game. For one layer at a time: record the intact model's full-vocabulary next-token distribution over twelve fixed English prompts; zero the smallest 5/10/20/30% of that layer's weights by magnitude (or add Gaussian noise); record the distribution again; score the Kullback–Leibler divergence between them; then restore the weights exactly and move on.

Nothing is written to disk — checkpoint hashes were verified unchanged after every run. Depth is reported normalised to 0–1, because the architectures have different depths and raw indices would not be comparable. Two quantities come out: fragility (how far the output moves per unit of damage) and concentration (what share of total damage the most sensitive fifth of the depth carries — 20% if information were spread perfectly evenly).

3 · The matched pair — same data, same fine-tune, one architecture apart

Bar chart: mean KL after damaging one layer. Z-Next 0.0077 base and 0.0128 after fine-tuning; the identically-trained transformer twin 0.0606 and 0.1171, that is 7.9 times and 9.1 times more fragile.

Worst single layer after fine-tuning: 0.0429 for us, 0.8796 for the twin. At that point the twin is not degrading, it is failing.

Horizontal bars: share of total damage carried by the most sensitive fifth of depth. Z-Next 37 and 39 percent, Qwen 41 and 52 percent, the transformer twin 72 and 69 percent, against a reference line at 20 percent for perfectly even spread.

The twin carries 72% of its total damage in a fifth of its depth. We carry 37%. That is the mechanism behind the fragility number: the twin has load-bearing layers and we appear not to.

The part we find most interesting is that fine-tuning barely moves it — 37→39 and 72→69. Whatever sets this is set by the end of pretraining, and supervised fine-tuning does not redistribute it. It is not a transient state.

What this means for evaluation practice. On the benchmark table those two models are 0.9 macro points apart. From the outside they look like the same model. From the inside they are not remotely the same model. An output-level eval cannot see this — not did not, but cannot, by construction.

4 · Against Qwen — and a result that goes against us

Our base model is the most robust arm we measured, on both pruning and noise. But here is a result that does not favour us, and it belongs in the table rather than in a footnote:

what fine-tuning does to fragility
  Z-Next      0.0077 -> 0.0128   (1.67x)   more fragile
  twin        0.0606 -> 0.1171   (1.93x)   more fragile
  Qwen        0.0185 -> 0.0103   (0.56x)   MORE ROBUST

Qwen's instruction-tuned model is more robust than its own base. Ours and our twin's both get more fragile. We cannot separate the two available explanations — Qwen's post-training pipeline is far more developed than ours, or an 18T-token base is simply too well-conditioned for fine-tuning to damage. Either way the conclusion for us is the same: there is something to learn on our fine-tuning side.

5 · The memory curve, stated more carefully than we stated it before

We have been quoting a seat-count ratio measured against our own twin. That number is real, but the twin is not KV-optimised — fp32, no grouped-query attention. A reader who knows the field would object that no modern deployment runs like that, and they would be right. So here is the same claim against a modern GQA/fp16 transformer.

Log-log chart of per-session memory against context length. Z-Next stays flat at 5.227 MiB from 512 to 32,768 tokens, while both transformer KV caches grow linearly; the ratio against Qwen is 5 times at 1k, 21 times at 4k and 171 times at 32k.

The correct claim was never a single number: our per-session cost does not depend on context length; theirs does. The ratio is not a constant, it is a curve, and it is unbounded in context. That is both more honest and harder to argue with, because it does not require us to pick the baseline.

6 · An unflattering measurement we are publishing anyway

The X-ray took 357 seconds on our model where it takes about ten on a comparable open transformer: 142 ms per forward pass versus roughly 15 ms. Our optimised inference path is gated off until it passes a parity test, so research measurements run on the reference implementation. Production inference does not use this path — the live demo is far faster — but every research measurement we take pays this bill.

7 · What this adds up to

Z-Next v4.3 is a model with a sound structure and a thin education. The things it lacks — world knowledge, retrieval from long context, conversational range — are limits we have traced to data, and data is bought with compute. The things it has — distributed representation, robustness to weight damage, a session state that does not grow — came from the architecture and do not improve with more tokens, because they are already there.

That is an unusual position to be in, and it is the reason we are looking for compute rather than for a new idea.

Reproduction

Every number here comes from two tools: our frozen benchmark harness, unchanged since 18 August and used unmodified for every anchor model, and the lesion X-ray described above. The X-ray protocol — including its reading rules and its declared failure condition ("if the arms are indistinguishable, that is a result and will be reported as one") — was written down before the runs.

We do not release Z-Next weights. We do release the numbers, the protocol, and the caveats, including the two results in this note that argue against us.

Z-Next architecture note · Live demo on one consumer GPU · Run this scan on your own model