Model X-Ray — Scope & Limitations
VG1 first beta · report schema rs-1.7 · presentation rp-1.3 · updated 22 September 2026 (Europe/Istanbul).
Current scope · 22 September 2026 · 7B class open
Current status. The 7B class opened on 21 September 2026. The eligible public repositories listed on the scope page (twenty-one exact Apache-2.0 revisions as of 22 September 2026, up to the 7B class) can be scanned by any registered account within the free allowance: 20 browser scans and five distinct source models per calendar month. 7B-class checkpoints run as single-model quantization simulations; a pair of two 7B-class checkpoints is currently accepted by the form but fails in the worker (under investigation, 22 September) — please do not submit one until this note is removed. Scans run on Tetracta-operated local GPU workers; no cloud GPU is used in this beta. A scan reads safetensors weights under their canonical file names; a repository that publishes only a pickle checkpoint or another format (for example GGUF) is declined before a job is created and does not use the allowance. Models above the 7B class are not scanned in this beta. External customer acceptance remains pending; results are research-stage and payments are disabled.
Registration page · Sample report gallery · Correction record
First-beta scope · external customer acceptance pending (15 September 2026 — superseded by the 22 September entry above)
VG1 is deployed. Internal end-to-end operational acceptance completed on 15 September 2026; external customer acceptance is still pending. The free first-beta scope covers the eligible public models listed below, up to and including 7B, within the account allowance. Customer jobs run on RunPod. (Superseded on 22 September 2026: jobs now run on Tetracta-operated local GPU workers; no cloud GPU is used in this beta.)
Models above 7B are outside this free first-beta scope. Later Pro access requires an explicit grant and remains limited to the models and allowance enabled for that account; registration alone does not grant access. (22 September 2026: models above the 7B class are not scanned in this beta; any later access to larger models will be announced with a dated update here.)
Eligible models
These are the public repositories the service is authorised to scan in this first beta. Public availability alone does not make a model eligible; each entry carries a completed rights review. The exact revision is pinned when a scan is submitted and is recorded in the report.
Updated 22 September 2026. The register now lists twenty-one repositories: six were added on 22 September (Mistral-7B-v0.3 and -Instruct-v0.3, SmolLM3-3B-Base and SmolLM3-3B, Qwen3-4B-Base and Qwen3-4B; all Apache-2.0, ungated, exact revisions pinned) and TinyLlama/TinyLlama_v1.1 was removed because it publishes no safetensors weights under the canonical names. The register review is dated 22 September 2026 and expires on 20 December 2026 unless renewed. A scan reads safetensors weights under their canonical names: model.safetensors, or model.safetensors.index.json with its shards. Repositories that publish only a pickle checkpoint (pytorch_model.bin) or only another format such as GGUF are declined before a job is created. Quantized release formats (for example FP8, GPTQ, AWQ or GGUF files) are not measured; the quantization simulation is derived in memory from the full-precision weights. Both checkpoints in a comparison must share the same architecture. A comparison keeps both snapshots on one worker and loads them one after the other. A pair of two 7B-class checkpoints is currently accepted by the form but fails inside the worker sandbox (under investigation, 22 September 2026); please do not submit one until this sentence is removed — a single 7B-class checkpoint can be scanned as a quantization simulation. The free allowance is 20 browser scans and five distinct source models per calendar month per account.
| Family | Repositories |
|---|---|
| Qwen2.5 | Qwen/Qwen2.5-0.5B · Qwen/Qwen2.5-0.5B-Instruct · Qwen/Qwen2.5-1.5B · Qwen/Qwen2.5-1.5B-Instruct · Qwen/Qwen2.5-7B · Qwen/Qwen2.5-7B-Instruct |
| Qwen2.5 · specialised | Qwen/Qwen2.5-Coder-1.5B-Instruct · Qwen/Qwen2.5-Math-1.5B-Instruct |
| SmolLM2 | HuggingFaceTB/SmolLM2-135M · HuggingFaceTB/SmolLM2-135M-Instruct · HuggingFaceTB/SmolLM2-360M · HuggingFaceTB/SmolLM2-360M-Instruct · HuggingFaceTB/SmolLM2-1.7B · HuggingFaceTB/SmolLM2-1.7B-Instruct |
| TinyLlama | TinyLlama/TinyLlama-1.1B-Chat-v1.0 |
| Mistral (added 22 September 2026) | mistralai/Mistral-7B-v0.3 · mistralai/Mistral-7B-Instruct-v0.3 |
| SmolLM3 (added 22 September 2026; listed in the register, but the current instrument release does not support this tokenizer contract — submissions fail; do not submit until this note is removed) | HuggingFaceTB/SmolLM3-3B-Base · HuggingFaceTB/SmolLM3-3B |
| Qwen3 (added 22 September 2026) | Qwen/Qwen3-4B-Base · Qwen/Qwen3-4B |
Twenty-one repositories as of 22 September 2026, all in or below the 7B class. Models above 7B are outside this free first-beta scope. A repository that is not listed here is declined before a job is created and does not consume your allowance.
What you can inspect
A before/after scan compares two supported artifacts under the same recorded measurement contract. A quantization simulation compares one source artifact with a simulated lower-precision state derived in memory. Neither result transfers automatically to another model, input set or execution environment.
Your HTML report keeps the observations and limits together. Retain it with your artifact revisions and task-evaluation results when reviewing a checkpoint change.
How to read the report
| Report item | What it tells you | Limit |
|---|---|---|
| Internal-response finding | Whether the versioned instrument decision rule observed a difference on the evaluated set. | No quality, safety or capability grade. No observed difference does not mean identical in every respect. |
| Output-text status | Whether the recorded generated token sequences changed under comparable output conditions. | Withheld means the output comparison could not be made; it does not mean unchanged or improved. |
| Artifact context | The recorded revisions and relevant configuration, tokenizer and generation differences. | Multiple differences can contribute. Do not attribute a combined artifact change to weights alone. |
| Depth view, when included | Categorical observations at evaluated positions, with unmeasured positions explicitly marked. | A response can propagate. This does not locate edited weights or establish a cause. If the difference is already present at the first measured point, the earlier boundary is unresolved. |
| Measured weight changes | How far each parameter group's weights moved from the reference: relative to the reference norm, and as an absolute change. | Read the relative and absolute columns together. The reference norm can differ a great deal between blocks (in reports 12 and 14 of the sample gallery it grows about 5.5 times from block 1 to block 32), so a relative change that falls with depth can reflect larger weights rather than where an edit landed. The table measures distance in the weights, not the response, and can order two changes that the response count does not; a larger weight change is not a quality or severity score, and neither column identifies the cause of a task outcome. Added 23 September 2026 after Dipankar Sarkar set the weight tables of reports 12 and 14 side by side. |
| Verify view | The finding, output status, context and available verification summaries. | Depth observations are not included in this view; the underlying finding and output status do not change. |
| Repeat, control and receipt summaries | Evidence actually included for the recorded run; absent items are labelled. | Do not infer a repeat, a signature, a noise envelope or a deletion guarantee from the report layout. |
| Recommended next checks | Guidance for choosing task evaluations and retaining a comparison record. | These recommendations are not additional measurements or a release decision. |
Coverage and comparison limits
- Not evaluated is not a zero. The historical unmeasured prefix was a scanner-coverage limitation, not a demonstrated sensitivity floor. Current categorical coverage makes that distinction explicit.
- Consistent contributors. The profile calculation holds its contributing set consistent across the evaluated region. It does not recover measurements absent from an older record.
- Recorded controls stay bounded. Repeat and same-artifact summaries describe their recorded runs. They do not establish agreement across every device, runtime, model family or task, or a population error rate.
- Keep comparisons comparable. Check the pair, revisions, measurement contract and runtime context before comparing reports. Do not turn descriptive observations into a ranking of unrelated models.
- No calibrated severity. A count of positions with an observed response is not the size of a weight edit, a measure of damage or an optimal training target.
Simulation and unavailable research modes
A quantization simulation does not produce a deployable artifact. It does not measure the memory, latency, numerical behavior or task quality of a packed inference runtime. Evaluate the exact deployed candidate separately.
Expect a saturated depth view in a quantization simulation. A simulation perturbs every quantizable weight tensor, so a depth view in which every evaluated position shows a difference — and in which the earlier boundary is therefore unresolved — is the expected shape of this scan type, not a finding about the particular model. This does not establish that every perturbed position is observable; the smallest change this instrument would miss is unmeasured. A saturated view also occurs in checkpoint comparisons where the change is not confined to part of the model.
Knowledge scans remain unavailable. Earlier knowledge-separation and fabrication-avoidance interpretations remain withdrawn. Hallucination, undertraining and overtraining are separate research questions; the beta report does not diagnose them. Historical portrait and lesion-response interpretations also remain withdrawn.
Access, rights and handling
- The service admits only supported public references and revisions with the required rights review. A model's public availability alone does not make it eligible.
- Private model uploads are unavailable in this beta. Do not submit private artifacts through a public model reference.
- The report delivers its findings and context. The instrument's private inputs, raw profiles and internal methods are not public report content.
- Read the handling terms at the point of use. This scope page makes no guarantee of erasure, continuous availability or model safety.
Versions and the correction record
The report identifies its instrument, schema, presentation and measurement contract. A new presentation or calculation does not repair missing measurements in a historical scan. The 12 September correction explains the retained evidence and what remains unresolved.
Read the historical scope text · 6–7 September 2026
Historical record, superseded where noted above. The original dated wording is retained below. In particular, the old “instrument floor” interpretation and seven-day larger-model estimate are not current claims. Its controlled experiments describe their own conditions and do not validate arbitrary customer models.
Historical scope text · 6–7 September 2026
Instrument release xray_vg1 · report schema rs-1.7 · measurement contract mv-1.4 / paired-artifacts-2.0 · last revised 7 September 2026 (Europe/Istanbul).
Update — 7 September 2026 (Europe/Istanbul). Onset depth. Since the 6 September release we ran a pre-registered control experiment in which changes of known size were introduced at known depths of three public models, and the instrument was asked where the difference begins. The criterion was frozen before any result was seen (pre-registration cd0171cf). Result: exact onset predicted and observed in 6 of 6 controlled cells on each of three public models (Qwen2.5-0.5B-Instruct, SmolLM2-1.7B, TinyLlama-1.1B-Chat-v1.0); cross-board agreement on onset 4 of 4 cells. On this basis, reports produced under instrument release xray_vg1 now state where a recorded difference first appears.
Onset depth is reported with the following calibration: the change first becomes observable at the output of decoder block k of N (blocks are numbered from 1). The first four profile positions — the embedding output and the outputs of blocks 1 to 3 — are below the instrument floor; the output of block 4 is the first evaluable position. Where the recorded difference is already present at that first evaluable position, no starting depth is reported: it may begin earlier, below the floor.
What this calibration does and does not cover. The control experiment used changes that we introduced ourselves, at depths we chose, in models whose weights were otherwise unchanged. It establishes that the instrument reports the depth we induced. It does not establish how the instrument behaves on changes that arise naturally, such as a fine-tune, where the change need not be confined to a known depth. It also does not measure the smallest change the instrument would miss: the published null control gives the rate at which identical pairs are reported as different (0 of 16), not the rate at which real differences go unreported. In the first pairs of this kind that we measured, the recorded difference was already present at the first evaluable position.
The finding is yours: every measured field in a Model X-Ray report — the verdict, where the difference begins, how far it extends, the recorded differences in configuration, tokenizer and generation settings, the output-text status, and the report's identity — is delivered in full, and you are free to publish or forward it.
What stays proprietary is the instrument, not the finding: the probe design, the instrument floor, the internal statistics and the transformations that turn a pair of artifacts into those fields.
We keep the instrument fixed and versioned, and we publish its null control, so that two reports taken under the same contract can be compared with each other — an instrument that changed between reports would make the findings you own worth less, not more.
Update — 7 September 2026 (Europe/Istanbul), Part 3. When a public model was fine-tuned by us on four consecutive decoder blocks at a known depth (LoRA r=8 on blocks LO..LO+3, merged; three depths), the reported onset matched the first fine-tuned block in 3 of 3 cells (blocks are numbered from 1). This does not establish behaviour on fine-tunes produced by others, whose depth and magnitude are unknown. Frozen B7 preregistration SHA-256: 1028eedb853fb662aeb85a6d5f0404df1dcf99d367088bc2c94a9000a3243977.
Two controls were run first: one artifact, re-serialized into a different shard layout with identical tensor values, was reported as no difference; a known lesion reported a difference at the expected depth. In 3 controlled micro-fine-tune pairs (single model family), every difference this instrument reported was also visible in the model's output text. We have not yet produced a case where the instrument reports a difference that a plain output comparison would miss. Until we do, we make no claim that it can. Frozen B8-SD preregistration SHA-256: d2dae32fa361f16a9f660241181b4a5bd19ca44338d284b15dbfc57e0c7a6d9d.
The reading tells you where a change first becomes observable. It does not tell you how large the change is, or what kind of change it was: in our controlled tests, a fine-tune and a structural lesion at the same location produced the same reading. Configuration is compared on a fixed set of fields; numeric-precision fields are not compared, and any file difference is recorded in the artifact identity.
In one line: onset localization works on our controlled fine-tunes; added value over plain output comparison is not yet shown.
Service status (7 September 2026, Europe/Istanbul): open beta. Any registered account may run up to 20 free scans on public models below 7B; models of 7B and above open in about one week. Release validation (V1 acceptance) remains open. (Superseded: the 7B class opened on 21 September 2026; see the current scope above.)
This page states what the instrument measures, what it does not, and where its blind spots are. It is the intended scope reference for current reports. If a report appears to exceed this scope, tell us.
What a scan is
A Model X-Ray scan compares two model artifacts (A and B, such as a base checkpoint and its fine-tuned version) under a fixed measurement contract. Its result applies to that pair and those recorded conditions.
The customer-facing result is deliberately narrow:
difference_observed— whether a difference was observed between the two artifacts under this contract.output_text_change_observed— whether the recorded output text differs between A and B. This field is withheld when their output contracts are not comparable.- Contributing probes — the number that actually contribute to the result, rather than the total number executed.
- Scope, versions, public artifact identities, and an operational receipt.
What the instrument does not claim
- Historical scope — 6 September 2026. The dated update above supersedes the former absence of an onset-depth field. Other limitations in this historical statement remain in force.
No localization. Earlier location and spread conclusions were withdrawn on 4 September 2026 and are not produced by rs-1.7. A replacement location claim requires a separate, preregistered evidence program. That claim remains pending validation.
- No knowledge, factuality or hallucination claims. The legacy knowledge evaluation did not isolate a model-knowledge effect. Its conclusions were withdrawn and are not part of this release.
- No portrait / lesion-response interpretation. Single-model portraits and lesion-response narratives remain withdrawn. No replacement conclusion is offered while validation is pending.
- No quality, safety, performance or deployment verdict. “Difference observed” does not mean worse, damaged or unsafe. “No difference observed” does not mean safe or unchanged in every respect. It means this instrument did not observe a difference under the stated contract.
- Simulated quantization is a simulation. Earlier simulated-quantization conclusions were withdrawn on 4 September 2026. Where an explicitly labelled simulation compares an artifact with its simulated counterpart, the result is descriptive only; it does not establish the behavior of a deployed quantized runtime, latency, or task quality.
Known blind spots and instrument properties
- Historical scope — 6 September 2026. The dated update above supersedes the former absence of an onset-depth field. Other limitations in this historical statement remain in force.
Detection floor. A detection floor is a property of the instrument, not evidence that a model changes at a particular location. Some changes can fall outside what the instrument can observe. This report makes no location claim.
- Fixed contributing set. The current contract uses a fixed contributing set within an eligible comparison. That restriction does not establish sensitivity to all kinds of changes. Coverage remains pending validation.
- Fixed comparison conditions. Results apply to the recorded release, artifacts and runtime. They are not representative of every possible input or operating condition. The measured A/A repeat results below apply only to their recorded card and runtime conditions.
- Numerical behavior. Numerical equivalence across GPU boards and runtime versions has not been established. A binary observation must be assessed from actual control results; its stability is not presumed. Private numerical profiles are not published.
- A/A null control. Before this release, and again whenever the runtime changes, the instrument is tested against the same artifact and revision on the supported board classes. The engineering control matrix completed 16 of 16 comparisons across four models (Qwen2.5-0.5B-Instruct, Qwen2.5-1.5B-Instruct, SmolLM2-1.7B, TinyLlama-1.1B-Chat; 0.5B–1.7B; two transformer architecture families): 0 observed differences and 0 output-text changes; no failed or unrun comparisons. The exact two-sided 95% Clopper–Pearson interval for observed differences is 0–20.59%. The 5070 Ti used PyTorch 2.11.0+cu128 / Transformers 5.10.1; the 4060 used PyTorch 2.9.1+cu128 / Transformers 5.5.4. On the RTX 5070 Ti, each model's two same-board runs produced byte-identical recorded metrics (4 of 4 model pairs; 8 runs). On the RTX 4060 class, each model's two runs were on two different physical cards and also produced byte-identical recorded metrics (4 of 4 model pairs; 8 runs); these are cross-board comparisons within one card class, not same-board repeats. Across the two card classes (different runtimes) the recorded measurement fields were identical while runtime identifiers differ; no cross-runtime byte equality is claimed. These 16 A/A comparisons are not 16 independent same-board repeat tests. The A/A design and decision rule were frozen before the first run: specification file sha256 7c9fb15b84657e67a688b164a5065116e01fab8dca8efee2d035759d32b44dc5, frozen 15:56 TRT on 6 September 2026; first A/A scan started 16:30 TRT. A non-zero observed-difference count would block use of the binary result pending a preregistered calibration study; the decision rule is not changed after seeing results. The observed rate describes the tested matrix, not a population error rate.
- Abstention. When artifact contracts are not comparable, affected fields are reported as withheld, not as “no change”. A failed or unrun comparison is not counted as a negative observation.
- Family and scale coverage. The completed A/A matrix covers Qwen2.5-0.5B-Instruct, Qwen2.5-1.5B-Instruct, SmolLM2-1.7B base, and TinyLlama-1.1B-Chat. Artifact-pair comparison coverage remains pending. Only model families, sizes and conditions actually measured will appear in the release evidence. Technical ability to load a model does not establish measurement validity for it.
- Input coverage. The fixed comparison conditions do not cover every language, task or behavior. Results must be read within one pair and must not be used to rank unrelated pairs.
How to read a report responsibly
- Compare A to B only. Do not compare numbers between two different reports.
- Treat “difference observed” as a prompt to run your own task evaluation on the change, not as a substitute for it.
- Treat “no difference observed” as “this instrument did not see it”, not as a certificate.
- Historical scope — 6 September 2026. The dated update above supersedes the former absence of an onset-depth field. Other limitations in this historical statement remain in force.
A calibrated risk grade, location, or model pass/fail verdict is not offered while the required evidence remains pending.
Data handling and rights
- Public Hugging Face references are identified at an exact revision; the revision is recorded in the report.
- Customer-uploaded processing remains unavailable until its operational checks pass. No upload availability or completed data-deletion assurance is claimed by this page.
- Only artifacts with a reviewed rights record for the intended use may be admitted. Unsupported license conditions remain held.
- Confidential measurement inputs, decision settings and raw response profiles are proprietary and are not disclosed in reports or public materials.
Versioning
Current reports identify their release, report schema and measurement contract. A contract change is versioned; results across different contracts must not be treated as interchangeable. Earlier findings remain withdrawn. Correction records retain their historical context.
Correction record: https://www.tetracta.ai/model-xray/correction/