Models / X-Ray twin study 1B

X-Ray twin study 1B (archived)

Archived study materials: two 0.93B language models trained under matched recorded conditions, one with standard softmax attention and one with our Rational Attention operator, plus three seed-control models. Their original use as evidence was withdrawn in September 2026; a bounded remeasurement with the current instrument followed on 12 September.

Status: Withdrawn as evidence
Updated: 11 Nov 2026
Licence: CC-BY-NC-4.0
Weights: Public, archived
Validation: Not validated; findings withdrawn

Status

On 4 to 6 Sep 2026 we withdrew all earlier public Model X-Ray findings as evidence, this study included. Read the Model X-Ray correction record

What the study claimed and what the seed control of 13 Jul 2026 found is under Evaluation records. For the product as it runs today, see Model X-Ray.

Model details

What the archive contains, as recorded on the Hugging Face card.
Item	Value
Developer	Tetracta
Type	Language models: pre-training checkpoints, with no instruction tuning
Arms	"vanilla": standard softmax attention. "rational": our Rational Attention operator, a drop-in replacement for softmax attention with zero extra parameters. The operator is not disclosed and is not in the repository.
Parameters	0.929B per model
Checkpoints	Steps 10,000, 20,000 and 30,000 for each arm, of a planned 157,000. float32 safetensors, input and output embeddings tied.
Seed controls	Two more softmax runs and one more rational run, published at step 30,000
Licence	CC-BY-NC-4.0
Status	Withdrawn as evidence; remeasured 12 Sep 2026
Contact	[email protected]

The "vanilla" checkpoints here are not VAN-1B. VAN-1B is a separate, fully trained 0.94B model.

Intended use

What the archive is still good for

Studying how a seed control can overturn an apparent effect.
Checking our historical training-loss records against the published checkpoints.
Teaching about negative controls in model comparisons.

Out of scope

Any use as evidence for Model X-Ray, for Rational Attention or for an architecture effect.
Benchmarking the rational checkpoints without the operator. Loading them into a softmax model gives meaningless output. Please do not report such results as "Tetracta rational".
Commercial use. The licence is non-commercial.
Deployment of any kind.

Training data

The original training record describes matched conditions for the two arms: the same seed, the same data order and the same schedule. The card does not name the corpus, and we add nothing here that the archive does not record. These training records have not been independently revalidated.

Training procedure

Both arms were trained under matched recorded conditions. The intended difference was the attention operator. Training stopped at step 30,000 of a planned 157,000, so these are young models.

The seed controls repeat the recipe with different seeds: two more softmax runs and one more rational run. All five checkpoints are sealed with md5 checksums.

Evaluation records

These are historical observations from the original training records. They are not new results, and they have not been revalidated.

Validation bits per byte at step 30,000. Spread between the three softmax seeds: 0.0020. Gap between the original softmax and rational models: 0.0010.
Model	Arm	Validation bits per byte
Original pair	softmax	1.0092
Seed control B	softmax	1.0072
Seed control C	softmax	1.0085
Original pair	rational	1.0102
Seed control B	rational	1.0101

The gap between the two arms is smaller than the spread between ordinary softmax runs. So the loss values are close for these checkpoints. That is not an equivalence test, and it does not show that the two designs are equal in general.

What was claimed, and what the seed control did to it

The study compared the insides of two twin models with an early Model X-Ray development pipeline and reported internal differences between them. It did not hold up.

On 13 Jul 2026 we ran the control we had flagged as the obvious next step: an independent-seed null. We trained three more models with different seeds and measured again, with thresholds sealed in advance. Every claimed internal difference between the twins fell inside the spread between ordinary runs of the same recipe. We retracted those claims.

The original internal claims and their status after the 13 Jul 2026 control. None of them is current evidence.
Original claim	After the seed control
The rational model's internals were 4.4 times quieter under int8 quantisation.	Retracted. The softmax seeds alone spread 7.3 times, and the softmax median sat below the rational models.
The rational model had a routing-stability edge under int8.	Retracted. The rational replication fell inside the softmax range.
The twins diverged internally across 84% of depth.	Retracted. Two softmax seeds diverged across the same depth by about the same amount.
The rational model abstained more on trick questions.	Already a null before the control: a generation-budget artefact (p about 0.2). Still a null.

Remeasurement on 12 Sep 2026

Five published step-30,000 checkpoints were scanned again with the current Model X-Ray instrument, as a bounded research run. Repeat scans of the same checkpoint showed no difference. Pairs across the two arms and pairs of seeds within one arm both showed differences. This does not establish an architecture effect, seed independence or a quality ranking, and it does not reinstate any withdrawn claim.

Limitations and what is not done yet

The models are young: step 30,000 of a planned 157,000.
Quantisation in the original study was simulated per-row round-to-nearest, not a production method such as GPTQ or AWQ.
Behavioural comparisons rested on 6 probe prompts.
One original pair and three seed controls are far too few to estimate a noise envelope or to draw population conclusions.
The rational arm cannot be run correctly without the undisclosed operator, which is not included.
Nothing here is a scaling law, and nothing here claims that Rational Attention produces a better model.

Not done: validation of the Model X-Ray method for claims of this kind, and any re-test of the original internal claims under a validated method. Our separate Rational Attention results are listed, with their own status, on the research page.

Risks and safety

The main risk is misuse as evidence. Do not cite these files, or the portrait images in the xray folder, as support for Model X-Ray, Rational Attention or any architecture effect.

Running the rational checkpoints in a softmax model produces meaningless output, and benchmarks run that way are invalid.

These are base pre-training checkpoints with no instruction tuning and no safety work. They are not suitable for any user-facing use.

Environmental impact

The card does not record GPU-hours for these runs, so we give no figure here. We have not measured energy use or emissions.

How to access

The archive is on Hugging Face: huggingface.co/tetracta/llm-xray-twin-study-1b.

Main paths in the repository. The seed-control checkpoints are described on the Hugging Face card.
Path	Contents
vanilla/	Softmax checkpoints at steps 10,000, 20,000 and 30,000, with config.json.
rational/	Rational checkpoints at the same steps, with config.json. They need the undisclosed operator.
xray/	Two portrait images from the withdrawn study. Archived, not evidence.
xray_summary.json, weight_error.json, reproduce_weight_error.py	Historical measurement summaries and a weight-error script. Archived.
manifest.json	sha256 checksums for the files.

The historical post-mortem stays online as a withdrawn citation anchor: twin study note.

Citation

@misc{tetracta_xray_twin_study_2026,
  title  = {LLM X-Ray twin study 1B: twin loss curves and the seed-null that retracted the headline (archived)},
  author = {Tetracta},
  year   = {2026},
  url    = {https://huggingface.co/tetracta/llm-xray-twin-study-1b},
  note   = {Archived study materials. Use as evidence withdrawn on 6 September 2026. See https://www.tetracta.ai/model-xray/correction/}
}

The Hugging Face card has no BibTeX entry. If you cite this archive, please cite its withdrawn status with it.

Changelog

Changes to the study and to this card.
Date	Change
Before 13 Jul 2026	First version of the card reported internal differences between the twins.
13 Jul 2026	Independent-seed null run on three more models. The internal-difference claims were retracted, and the card was rewritten around the retraction.
17 Jul 2026	Historical update note added. It has since been superseded.
6 Sep 2026	All earlier Model X-Ray findings withdrawn as evidence, this study included.
12 Sep 2026	Bounded remeasurement of five published checkpoints with the current instrument. It does not reinstate any withdrawn claim.
8 Nov 2026	This archived study card published on tetracta.ai.
11 Nov 2026	Status wording brought up to date. No result changed.

Ask about Model X-Ray

What the beta can measure today, and what is still being validated.

Ask a question