We spent three weeks trying to kill our own finding. Here’s what survived.

“The change starts deeper in bigger models.”

Early scans suggested quantization damage “starts” at a layer that grows with model size. Before claiming it we ran a control: we injected damage only into the first four layers of a model and asked the scanner where the change began. It said layer 4. The instrument has a floor — it cannot see shallower — and our “absolute onset” was partly the floor talking. That claim went in the bin. And we kept pulling: a reviewer asked whether the slow crawl of onsets across sizes (4 → 5 → 5 → 6 → 7) was itself just the floor rising in bigger models. So we measured the floor at 32B directly — and it had risen, to 6, not 4. The onset sits a near-constant one layer above the floor at every scale; the “crawl” is mostly the floor moving, not a deepening effect. What actually survived is narrow and true: the quantization onset is always just above the instrument’s floor — no dramatic scaling law, and we said so before anyone made us.

The width metric didn’t survive its own control.

Our first “how wide is the change” number failed its own quantization control. We rebuilt it (report-level definition: the share of layers carrying 80% of the difference mass), re-ran every scan, and republished the corrected series. The contrast got smaller and more defensible.

The indexing confession.

A layer-indexing convention (whether you count the embedding stage as a station) shifted several headline numbers by one. Nobody would ever have caught it from outside. We disclosed it anyway and re-baselined, because a diagnostic product whose numbers move silently is worthless.

Is the “band” even real, or just geometry?

Deep-layer signals in transformers are simply bigger. Maybe our “concentration band” was just norm-geometry wearing a costume. So we re-analyzed every pair in normalized space. Result: the band’s position is indeed partly geometry — we say so in every report now. But the contrast between fine-tune and quantization survived normalization at all four scales we tested. The claim that survived is stronger for what it lost.

The day our product accused a correct label.

Our reports include a declared-vs-observed check: you tell us what you did to the model, the scan says whether the internal signature matches. On a 32B fine-tune — correctly declared — the engine returned MISMATCH. It was wrong. The signature bands it had learned from smaller models simply don’t transfer in absolute terms to 32B. We left that wrong verdict in the report, visibly, because it taught us the design rule the next version is built on: judge by contrast, not by absolute position. A system that hides its own misfires can’t sell you honesty about yours.

The cross-family test — four families, one honest scoreboard.

Everything above lived inside one model family (Qwen2.5, 0.5B → 32B). So we took the strongest claim to families that owe us nothing. Pre-registered predictions, official weights, hash-verified where licenses allow.

The raw-space contrast broke in Llama. In Llama-3.1-8B the fine-tune signature is far more diffuse than in any Qwen model, and one of our two locked predictions failed outright. The headline claim is therefore scoped: within Qwen2.5. The instrument itself transferred — at matched scales: the detection floor came out identical (layer 4) in Qwen 0.5B, Qwen 7B and Llama 1B; the probe is family-independent there while the signatures are family-specific. (The floor is not scale-invariant — it rises to 6 at 32B, as Attempt 1 records.) That probe-universal / signature-specific distinction is the spine of the product.

Then a door opened — and we measured its margin before walking through. In normalized space the contrast held on a second Llama scale (3B) and pointed the same way in a third family (Mistral-7B). But a headline needs more than a direction; it needs a margin bigger than the noise. So we measured the test-retest band of the difference metrics themselves (9 probe seeds) — and the verdict trimmed our own claim: the normalized contrast clears the band in two families (Qwen ×4 scales, Llama ×2); Mistral stays inside the band — a consistent direction, not evidence. We log it as inconclusive.

So we ran the fourth family — and it missed. Gemma-2-2B, predictions locked before looking. The two locked conditions had to hold together in normalized space; one came back a flat tie (effective width identical between fine-tune and quantization, to the third decimal), and in raw space the width even pointed the wrong way, by more than the noise band — a sharper counter-example than Llama ever handed us. Only the enrichment half of the signature cleared the band, as some version of it now has in every family we’ve touched; whether that is the portable component is a new hypothesis, and it goes through the same pre-registration gauntlet before we claim anything. “Normalized space is the portable form” remains what it always was — a hypothesis, now with a scar. (One caveat cuts in Gemma’s favor: its instruct checkpoint is trained with heavy distillation, so its “fine-tune” is not process-matched to the other families’ — recorded, not excused.) We publish the miss for the same reason we publish everything else: a claim that only ever gets stronger is a claim nobody is actually testing.

The honest headline: the normalized fine-tune-vs-quantization contrast holds, band-exceeding, in the two families we tested it clears — and we publish the family where it didn’t.