Model merging · measured · failures included

Chimera-1: we stitched two specialists into one model — training-free — and measured everything, including what failed.

Tetracta AI Teams · 22 July 2026 · model on Hugging Face · ~10 experiments, all on two local consumer GPUs, $0 cloud

A chimera, in biology, is one organism carrying two genetic lineages — which is exactly what this model is. Model merging is everywhere — leaderboards are full of merged models, and the recipes are mostly folklore: merge blind, eval after, keep what sticks. We wanted the other thing: when does merging actually create capability, when does it destroy it, and how much of that is predictable from the weights before you merge? So we ran the whole arc on our own hardware and wrote down every number, including the embarrassing ones.

+13 / +26points on the two target skills, in ONE model
0measured general regression (Belebele-EN, perplexity, chat)
$0cloud cost — merge runs on CPU in minutes
3guided-merge attempts that did NOT beat blind TIES (we say so)

The result

Start from Qwen2.5-3B-Instruct. Make two light specialists: one for Roman-numeral conversion (98% on task), one for character-level word operations (75%). Then merge them with TIES — task-vector trim, sign election, disjoint mean — a procedure with no training step that runs on a CPU in minutes:

Bar chart: base model scores 76% on Roman numerals and 45% on letter operations; specialist A reaches 98% Roman but stays 45% letters; specialist B 80% Roman and 75% letters; the merged Chimera-1 reaches 89% Roman and 71% letters in a single model
One model, both skills: 89% / 71% vs the base's 76% / 45%. Retention checks came back identical to the base — Belebele-English 50%=50%, fixed-text perplexity 68.1=68.1, chat outputs indistinguishable.

The point is not Roman numerals — the skills are deliberately simple, verifiable testbeds. The point is that a single 3B model carried both parents' gains at zero extra inference cost and zero measured general damage, with no training at the merge step. The model is public; the load-and-try snippet is on the Hugging Face card.

The law: when merging works at all

Across ~10 experiments (0.5B and 3B, same-family and cross-family, our own specialists and off-the-shelf ones), one rule kept deciding the outcome: synergy = headroom × complementarity × proximity.

  • Headroom. If the base already solves the task, there is nothing to merge for. Our first 3B attempt died exactly this way (base at 100%/83% on the toy tasks — merge had nothing to show).
  • Complementarity. Parents must each be genuinely stronger somewhere. Merging a generalist with a coder that was no better at code than the generalist just diluted both (−8 code, −11 multilingual).
  • Proximity. Parents must still live near the shared base. Aggressively merging a heavily continued-pretrained coder produced mush: 0% on code, chance on everything else — our cleanest negative control.

The honest part: what did not work

We build an internal-measurement instrument (an X-Ray for LLMs), so the obvious ambition was: use per-layer skill maps to guide the merge and beat the blind recipe. Three attempts — divergence-weighted scaling, a per-layer conflict referee, a finer per-module referee with a do-no-harm default: none of them beat uniform TIES on average (the best tied, 80%=80%, with a different balance: 95/65 vs 89/71). Blind TIES is a genuinely strong baseline, and we are not going to pretend otherwise.

What internal measurement did reliably deliver, in every run: it predicted before merging whether a pair would merge profitably — headroom, complementarity and proximity are all measurable from the weights and activations up front (3/3 correct calls in this campaign, including the mush case) — and the guided variants act as a balance dial, letting you choose which parent's skill to protect. Diagnosis first; surgery only when it provably beats the blind knife.

Bonus: keep both brains, route between them

Weight-merging requires a shared base. For models with different tokenizers and architectures (Qwen2.5-3B + Llama-3.2-3B) we ran the other family of fusion: keep both models intact and, at every token, generate from whichever is more confident — after correcting for the fact that one model is systematically more overconfident than the other (0.902 vs 0.870 mean top-probability).

Bar chart of routing strategies on 143 hard questions across 7 languages: Qwen alone 75%, Llama alone 73%, confidence-routing variants 76-78%, de-biased confidence routing 79%, oracle ceiling 84%
143 hard questions, 7 languages: every routing variant beats the best single model; de-biased confidence routing captures about half of the available oracle gap (79% vs 75%, ceiling 84%). Cost: you run two models.

Update (22 Jul): the full head-to-head — 1,210 questions

After release we ran the deep measurement: Chimera-1 against its base on the same LL-based harness, across MMLU (200), ARC-Challenge (200), HellaSwag (150), Belebele in five languages (500) and the two implanted skills (160). Sanity check first: the base scores MMLU 67.0%, matching its publicly reported numbers — so the harness measures correctly.

Horizontal delta chart comparing Chimera-1 to its base across ten benchmarks: MMLU minus 0.5, ARC-Challenge 0.0, HellaSwag plus 1.3, five Belebele languages between minus 1.3 and 0.0 — all inside the plus-minus two point noise band — while Roman numerals rises 8.8 points and letter operations rises 25 points
The surgical profile. Two implanted skills moved (+8.8, +25.0); all eight standard benchmarks stayed inside the ±2pp bf16 run-noise band (max |Δ| = 1.3pp). “Zero collateral damage” graduated from a 40-question retention check to a 1,050-question verification.

This is the result we consider the real headline — not that the merge is spectacular, but that it is clean: a capability edit with a measured blast radius of zero.

Method (summary)

Base: Qwen2.5-3B-Instruct. Specialists: top-8-layer light fine-tunes on synthetic, infinitely-variable tasks (700 examples each; specialists verified on-task before merging). Merge: TIES — task vectors vs the shared base, top-20% magnitude trim, sign election, disjoint mean; no training, CPU. Evaluation: 80 held-out items per skill, greedy decoding; retention via Belebele-English (40 items), fixed-text perplexity, and side-by-side chat outputs. Run-to-run bf16 jitter is about ±2 points; we do not interpret differences at that scale. Routing experiment: per-token confidence comparison with warm-up de-biasing; 143-question multilingual bank, adversarially validated. The internal skill-mapping instrument and fine-tune hyperparameters are ours; the merge procedures are standard and public.

Chimera-1 on Hugging Face → The X-Ray instrument

License note: inherits the Qwen Research License from its base (non-commercial, research use).

← All research notes

— Tetracta AI Teams · for humans, like humans. We publish the failures too.