VAN-1B

Model details

VAN-1B is a deliberately ordinary transformer. We built it as a control. It was trained on the same corpus, with the same tokenizer and a similar token budget as Z-Next 1B, and it is measured with the same harness. That lets us say where Z-Next stands against a transformer that saw the same data.

It was produced end to end by our LLM training pipeline: data collection and cleaning, corpus and tokenizer build, pre-training, instruction tuning, benchmarks, a blind judge panel and the Hugging Face release.

VAN-1B is not the same model as the "vanilla" checkpoints in the archived X-Ray twin study. Those are separate, shorter training runs of 0.929B parameters.

Intended use

Primary uses

• Research on data efficiency and instruction tuning at the 1B scale.
• A same-data transformer baseline when you compare architectures on this corpus.
• Teaching: a complete small model whose training is documented from data to release.
• Tool routing inside a program that checks the tool results and renders them itself.

Out of scope

• Anything where a wrong fact or a wrong number matters.
• Unsupervised use, including as a public chat assistant.
• Languages other than English.
• Long-form writing, multi-step reasoning and arithmetic.

Training data

Pre-training started from about 12.5B raw tokens of English text. After filtering, which includes adult-content and spam filters, and near-duplicate removal with MinHash, 10,321,920,000 tokens remained. The model saw them once: one epoch, no repetition.

This is the same corpus and tokenizer that Z-Next 1B was trained on.

Instruction tuning for the published model used 11,419 examples, our second tuning round. Part of this data was produced with the help of other AI models or by translation, and its provenance and usage terms are not fully documented.

Apache-2.0 covers our weights and code; it grants no rights in third-party material.

Training procedure

Evaluation

Language modelling

On held-out text the base model reaches 3.7610 bits per token, a token-level perplexity of 13.56. This is an older metric. It cannot be compared with the validation perplexity of 15.1 reported for Z-Next 1B, which uses a different metric and setup.

Ten zero-shot benchmarks

LAMBADA, HellaSwag, PIQA, WinoGrande, ARC-Easy, ARC-Challenge, OpenBookQA, SciQ, BoolQ and MMLU: full sets, zero-shot, macro average in percentage points. We measured every row ourselves, with the same harness and the same 1,024-token ceiling. Intervals are 95% paired bootstrap intervals over per-item correctness, with 4,000 resamples.

Read these numbers with four caveats. First, Z-Next and VAN-1B share the corpus and tokenizer but not an identical training recipe: a small share of Z-Next's pre-training windows also carried program-generated material that the transformer did not see. The level base result therefore shows no quality cost for Z-Next; it does not show an architectural advantage. Second, the two models are not matched layer for layer, so the comparison covers architecture and shape together. Third, Pythia was trained on a different corpus (the Pile). We measured its public checkpoints at step5000, 10.49B tokens, to match the budget; only the token budget and the harness are equal. Fourth, there is one run per model and one seed.

Most of what instruction tuning cost VAN-1B sits in two tasks: LAMBADA fell 8.50 points and BoolQ fell 5.63 points. The same recipe raised Z-Next by about 0.5 points, so most of the gap between the two tuned models comes from VAN-1B dropping.

Blind judge panel

64 prompts in 8 lanes, judged blind by 24 model judges, for 192 decisions. The score is the instruction-tuned VAN-1B's share of the decisions against each opponent.

Against the Qwen2.5-1.5B base model, VAN-1B is close to even, with its wins in the identity, honesty and chat lanes. Against the instruction-tuned models it is clearly behind. The card lists its losses in knowledge, reasoning and long-form writing, including a writing lane of 0 wins in 8.

What instruction tuning changed

Tuning made the model far more decisive and raised its plain-text loss. In a small tool-use check, the tuned model made 4 correct tool calls out of 6.

Limitations and what is not done yet

• Reasoning and arithmetic are not reliable. Multi-step problems collapse.
• Factual accuracy is weak. It invents rare or detailed facts.
• It can corrupt digits when it copies a tool result into its answer. Render tool results in code instead of asking the model to repeat them.
• Without the tool runtime attached, it may invent live data.
• It is inconsistent about its own abilities.
• English only. Long-form writing is clearly behind larger models.
• The shipped files support a context of 1,024 tokens.
• The evaluations rest on one run with one seed, and the judge panel uses model judges only.

Not done yet: a human evaluation, a formal safety evaluation or red-team, and repeat training runs with other seeds.

Risks and safety

The main risk is a fluent, confident wrong answer. A model of this size has no reliable sense of which of its recollections are solid. Do not put it in front of users without a program that checks what it says.

In place: the pre-training data passed adult-content and spam filters, and the judge panel includes a safety lane. That lane is a model-judged comparison; no safety evaluation has been run.

Not in place: RLHF, red-teaming and an output content filter. None of these is on record for VAN-1B. Apache-2.0 permits commercial use, so whoever deploys the model is responsible for checks that fit their use.

Environmental impact

We have not measured energy use or emissions for these runs.

How to access

Weights, tokenizer and code are on Hugging Face: huggingface.co/tetracta/tetracta-van-1b.

Embeddings are tied, so head.weight is intentionally absent from the safetensors files. Loading code and a usage example are on the Hugging Face card.

Citation

@misc{tetracta_van1b_2026,
  title  = {Tetracta VAN-1B: What does SFT do? Measuring a 0.94B model from the inside and the outside},
  author = {Tetracta},
  year   = {2026},
  url    = {https://tetracta.ai}
}

This is the entry from the Hugging Face card. To cite the weights directly, add the repository URL.

Changelog