Model merging · measured · failures included

Chimera-1: we stitched two specialists into one model — training-free — and measured everything, including what failed.

Tetracta AI Teams · 22 July 2026 · model on Hugging Face · ~10 experiments, all on two local consumer GPUs, $0 cloud

A chimera, in biology, is one organism carrying two genetic lineages — which is exactly what this model is. Model merging is everywhere — leaderboards are full of merged models, and the recipes are mostly folklore: merge blind, eval after, keep what sticks. We wanted the other thing: when does merging actually create capability, when does it destroy it, and how much of that is predictable from the weights before you merge? So we ran the whole arc on our own hardware and wrote down every number, including the embarrassing ones.

+8.8 / +25.0points on the two target skills (deep eval), in ONE model
0reported general checks outside the prespecified ±2-point band
$0cloud cost — merge runs on CPU in minutes
3exploratory variants that did not beat the TIES baseline

The result

Start from Qwen2.5-3B-Instruct. Make two light specialists: one for Roman-numeral conversion (98% on task), one for character-level word operations (75%). Then merge them with TIES — task-vector trim, sign election, disjoint mean — a procedure with no training step that runs on a CPU in minutes:

Bar chart: base model scores 76% on Roman numerals and 45% on letter operations; specialist A reaches 98% Roman but stays 45% letters; specialist B 80% Roman and 75% letters; the merged Chimera-1 reaches 89% Roman and 71% letters in a single model
One model, both skills: 89% / 71% vs the base's 76% / 45%. Retention checks came back identical to the base — Belebele-English 50%=50%, fixed-text perplexity 68.1=68.1, chat outputs indistinguishable.

The point is not Roman numerals — the skills are deliberately simple, verifiable testbeds. The point is that a single 3B model carried both parents' measured gains at zero extra inference cost, with no detected regression outside the prespecified ±2-point band on the reported general checks, with no training at the merge step. The model is public; the load-and-try snippet is on the Hugging Face card.

An observed pattern in this campaign

Across ~10 exploratory experiments in this campaign (0.5B and 3B, same-family and cross-family, our own specialists and off-the-shelf ones), headroom, complementarity and proximity appeared associated with the observed outcomes. We treat synergy = headroom × complementarity × proximity as a campaign-specific working hypothesis, not a validated predictive rule or general law.

  • Headroom. If the base already solves the task, there is nothing to merge for. Our first 3B attempt died exactly this way (base at 100%/83% on the toy tasks — merge had nothing to show).
  • Complementarity. Parents must each be genuinely stronger somewhere. Merging a generalist with a coder that was no better at code than the generalist just diluted both (−8 code, −11 multilingual).
  • Proximity. Parents must still live near the shared base. Aggressively merging a heavily continued-pretrained coder produced mush: 0% on code, chance on everything else — our cleanest negative control.

Revision: guided interpretations withdrawn

Three exploratory merge variants did not beat the uniform TIES baseline on average. The former article also interpreted internal measurements as predicting merge success and guiding skill retention. Those Model X-Ray interpretations are withdrawn and are not part of the retained merge result. The public result on this page is limited to merge measurements evaluated by Tetracta separately from Model X-Ray, the baseline comparison, general checks and negative control. This is not an external independent review. See the current Model X-Ray evidence boundary.

Bonus: keep both brains, route between them

Weight-merging requires a shared base. For models with different tokenizers and architectures (Qwen2.5-3B + Llama-3.2-3B) we ran the other family of fusion: keep both models intact and, at every token, generate from whichever is more confident — after correcting for the fact that one model is systematically more overconfident than the other (0.902 vs 0.870 mean top-probability).

Bar chart of routing strategies on 143 hard questions across 7 languages: Qwen alone 75%, Llama alone 73%, confidence-routing variants 76-78%, de-biased confidence routing 79%, oracle ceiling 84%
143 hard questions, 7 languages: every routing variant beats the best single model; de-biased confidence routing captures about half of the available oracle gap (79% vs 75%, ceiling 84%). Cost: you run two models.

Update (22 Jul): the full head-to-head — 1,210 questions

After release we ran the deep measurement: Chimera-1 against its base on the same LL-based harness, across MMLU (200), ARC-Challenge (200), HellaSwag (150), Belebele in five languages (500) and the two implanted skills (160). Sanity check first: the base scores MMLU 67.0%, matching its publicly reported numbers — a sanity check consistent with the public reference value.

Horizontal delta chart comparing Chimera-1 to its base across ten benchmarks: MMLU minus 0.5, ARC-Challenge 0.0, HellaSwag plus 1.3, five Belebele languages between minus 1.3 and 0.0 — all inside the plus-minus two point noise band — while Roman numerals rises 8.8 points and letter operations rises 25 points
The measured profile. Two target skills moved (+8.8, +25.0); all eight reported standard benchmarks stayed inside the prespecified ±2pp band (max |Δ| = 1.3pp). This supports only “no detected regression outside that band” on these checks.

This is the result we consider the real headline — not that the merge is spectacular, but that it is bounded: target-task gains with no detected regression outside the prespecified ±2pp band on the reported general checks.

Method (summary)

Base: Qwen2.5-3B-Instruct. Specialists: top-8-layer light fine-tunes on synthetic, infinitely-variable tasks (700 examples each; specialists verified on-task before merging). Merge: TIES — task vectors vs the shared base, top-20% magnitude trim, sign election, disjoint mean; no training, CPU. Evaluation: 80 held-out items per skill, greedy decoding; retention via Belebele-English (40 items), fixed-text perplexity, and side-by-side chat outputs. Run-to-run bf16 jitter is about ±2 points; we do not interpret differences at that scale. Routing experiment: per-token confidence comparison with warm-up de-biasing; 143-question multilingual bank, adversarially validated. Private fine-tune details are not disclosed; the merge procedures are standard and public. No Model X-Ray result is used as evidence in this method summary.

Chimera-1 on Hugging Face → Model X-Ray evidence status

License note: inherits the Qwen Research License from its base (non-commercial, research use).

← All research notes

— Tetracta AI Teams · for humans, like humans. We publish the failures too.