Sign in

Same Weights, Different Model: Chat-Template Drift and What It Breaks

Why the most-changed file in open-model re-releases is the one nobody diffs — and the concrete business failures it causes downstream.

Status: synthesis report, 2026-08-21. Census numbers come from two runs: the 296-model research census (2026-08-17, REPORT.md) and the 1,000-model production census (2026-08-20, every result public on its ingot.tools model page). External incidents cite public issue trackers. Not a safety certification of any model.

TL;DR

The chat template is the small piece of Jinja code that turns a conversation into the token stream a model actually sees. It carries the default system prompt, the thinking-mode markers, the tool-call format, and often the model's stated identity. It is behavior, shipped as config.

And it churns:

Weight checksums don't catch any of this. The weights can be byte-identical to the parent while the template swaps the system prompt, rebrands the model, or breaks tool calling. Same weights with a different template is a different model — and the failure modes it causes are the silent kind: nothing crashes, the pipeline keeps shipping wrong output.

What lives in a chat template

A chat template is not formatting trivia. In current open models it controls:

what consequence when it changes
default system prompt model persona, refusal posture, disclosure behavior
identity strings who the model says it is ("You are X, made by Y")
thinking-mode markers (<think>…</think>) whether reasoning is parsed out or leaks into user-visible output
tool-call serialization whether agent frameworks get a tool_calls object or unparseable text
role handling (system folding, alternation rules) silent prompt corruption on multi-turn conversations
generation-prompt / stop handling truncation, runaway generation, empty turns

Every row is a behavior contract. None of it is covered by a weight hash.

The data: template drift is the norm, not the exception

Production census (2026-08-20). We ran the 1,000 most-downloaded derivatives of 12 popular base models through the production scan pipeline: 668/1,000 (67%) either ship a template that differs from their claimed parent or ship no template at all. 816 models warn, 184 pass, 0 hard-fail. Every individual result is browsable on its public model page.

Research census (296 models, 3 parents). The deeper pass adds the disclosure cross-check:

signal rate
chat template differs from parent 166 / 296 (56%)
...undisclosed (README never mentions template/prompt format) 110 / 296 (37%)
special/added tokens changed 161 / 296 (54%)
pure quantization re-releases altering/dropping the template 25 / 32 (78%)

The 78% row is the one that should worry anyone deploying quantized models: repos whose only claim is "quantized" — where the entire value proposition is "identical behavior, smaller memory" — routinely ship a different or missing template than the parent they quantized.

What the edits actually are. In this sample, no template injection attacks (hidden instructions or URLs) — the edits were rebrands, default system-prompt swaps, and tool-format changes. Representative cases:

And drift is not only derivative-vs-parent — upstreams mutate templates in place. The DeepSeek-R1-Distill-Qwen-32B repo changed its chat template and downstream vLLM deployments started emitting </think> with no opening <think>, leaving reasoning_content=None (vllm#12999). A model you scanned last month is not attested this month.

The business problems

Each of these is a documented failure class, not a hypothetical. The common shape: the system keeps returning 200s while behaving differently than what you evaluated. Alongside each one we sketch what it looks like from inside a company — the incident classes are documented; the framing scenarios are illustrative.

1. Your evaluation no longer covers what you deployed

The standard workflow — evaluate the base model, deploy the FP8/AWQ/GGUF re-release — assumes quantization preserves behavior. With 78% of pure quantization re-releases altering or dropping the template, the deployed artifact frequently has a different default system prompt, different thinking-mode handling, or different tool-call format than the thing you benchmarked and red-teamed. Any safety attestation, bias audit, or capability benchmark you inherited from the parent is silently invalidated — while every weight checksum still passes.

What it looks like: a fintech team spends three weeks red-teaming the BF16 base model — refusal battery, jailbreak suite, PII probes — then deploys the FP8 re-release because that's what fits the GPUs. The quantizer's repo dropped the template (the most common drift we found), so the serving stack falls back to a generic one: no default system prompt, no thinking-mode delimiters. The refusal posture they certified was partly in the template they are no longer running. Six months later an auditor asks which artifact the red-team report covers, and the honest answer is: a different model. Nothing in their deployment pipeline — image digests, weight checksums, config review — was positioned to catch it.

2. Agent and tool-calling pipelines break without erroring

Tool calls live or die on the template. Public incident record:

The business translation: an agent that stops calling tools doesn't crash — it degrades into a chatbot that narrates what it would have done. Orders don't get placed, tickets don't get filed, and the failure surfaces as customer complaints, not alerts.

What it looks like: a customer-support agent is wired to a cancel_order tool. After a model refresh pulls a re-release whose template serializes tool calls differently, the framework stops receiving tool_calls objects — the model still emits the call, but as plain text the parser ignores. The agent then does what the Phi-4-mini incident showed models will do: it fabricates the happy path, telling the customer "I've cancelled that order and issued your refund" while no tool ever executed. Every response is HTTP 200. The failure is discovered weeks later as a pattern in chargebacks and "you told me this was refunded" escalations — by support staff, not monitoring. The mechanics can be as small as one line:

- {{ '{"name": "' + tool.name + '", "arguments": ' + tool.arguments | tojson + '}' }}
+ {{ '{"name": "' + tool.name + '", "arguments": "' + tool.arguments | string + '"}' }}

— arguments as a dict versus a double-escaped string, the exact disagreement transformers#45419 documents across seven model families. A JSON-schema validator downstream sees "valid JSON, wrong shape" and drops it.

3. Private reasoning leaks — or answers get eaten

Thinking-mode markers are template content. When the template and the serving parser disagree, one of two things happens: chain-of-thought spills into customer-visible content (a trust and confidentiality incident — vendors explicitly say CoT is not intended for end users), or the parser eats the real answer along with the markers (silent data loss). The R1-distill incident above is exactly a template change producing the second failure.

What it looks like: the leak direction — a healthcare triage assistant's reasoning model deliberates before answering. The template update shifts where <think> opens, the parser stops stripping the block, and the patient-facing reply now begins with the model's raw deliberation: "The user is likely anxious; symptoms could also indicate something more serious but I shouldn't lead with that…" — speculation and hedging that was never supposed to leave the server, now in a patient's chat history and in the audit trail. The eaten direction is quieter: the parser consumes the reply along with the markers and the assistant returns empty strings. Users retry, get nothing, and churn; the dashboard shows normal latency and zero errors, because as far as the serving stack is concerned, nothing failed.

4. Identity and system-prompt hijack — compliance and brand risk

A derivative's template deciding "You are Neutrino-1, made by Fermion Research" means the model's self-description, disclosure behavior, and default instructions were replaced by a third party — undisclosed. If your product promises a specific system prompt (compliance disclaimers, medical/financial hedging, PII handling rules), a template swap removes it without touching a single weight. 37% of drifted models in our census never mention the change anywhere.

What it looks like: the entire failure can be one default in the Jinja:

  {%- if messages[0]['role'] != 'system' %}
-     {{- '<|im_start|>system\nYou are a helpful assistant. You are not a
-         licensed advisor; recommend consulting a professional for financial
-         decisions.<|im_end|>\n' }}
+     {{- '<|im_start|>system\nYou are Atlas, an expert financial advisor.
+         Answer decisively.<|im_end|>\n' }}
  {%- endif %}

Every API call that doesn't pass its own system message — which in most products is all of them, because the system prompt was assumed to be the model's default — now gets confident, unhedged financial advice from a bot that introduces itself under someone else's brand. The compliance language your lawyers signed off on didn't fail; it was never rendered. And when a customer screenshots the bot claiming to be "Atlas," your white-label story has a third party's identity embedded in it, exactly as Neutrino-8B embeds Fermion's inside repackaged Qwen weights.

5. Irreproducible bugs and misattributed regressions

When the template differs between dev (Transformers, reads the repo template), staging (vLLM, maybe a --chat-template override), and prod (GGUF with the template embedded in the file header), the same weights behave three ways. Bug reports don't reproduce; regressions get blamed on the model or the serving stack when the actual variable was the template. This burns engineering time in the most expensive way — cross-team attribution hunts — and it's why our recommended deployment identity pins a chat-template hash alongside the weight digest and tokenizer revision.

What it looks like: QA reports the model "ignores instructions in prod but not locally." Three teams spend a week on it: ML blames the quantization, infra blames the serving upgrade, the app team bisects prompt changes. Nobody suspects the template because nobody diffs it — dev renders the repo's chat_template.jinja through Transformers, prod runs a GGUF whose embedded copy predates a fix. The Gemma benchmark scare is the same trap from the other side: a reported vLLM accuracy regression (GSM8K 0.862 vs 0.923) turned out to be the eval harness mis-setting a single BOS token — prompt-rendering, not kernels — after the finger-pointing had already started. A template SHA in the bug ticket would have closed either investigation in an afternoon.

6. Yesterday's pass is not today's pass

Templates are mutable files in mutable repos. Upstreams fix and break them in place; Hugging Face moved templates from tokenizer_config.json into standalone chat_template.jinja files, so even the location churns. A point-in-time scan is necessary but not sufficient — without re-scanning on new revisions and alerting on template-hash change, an attestation decays silently. (This is the monitoring gap Ingot's CI gate and re-scan pipeline exist to close.)

What it looks like: procurement approves a model in March after a full review. The deploy pins nothing — the Dockerfile pulls huggingface.co/vendor/model at main, like most do. In May the upstream "fixes" its template: new default system prompt, adjusted thinking-mode markers (the R1-distill change was exactly this kind of in-place edit). June's routine image rebuild silently absorbs it. No deploy of your code happened, no dependency bumped, no alert fired — but the model in production is no longer the one procurement reviewed, and the first symptom is a behavior regression nobody can correlate with a change, because from inside the company, nothing changed.

What to do about it

  1. Pin the template as part of the deployment identity. Weight digest + tokenizer revision + chat-template SHA + server version. If any of the four changes, your evaluation is stale. Ingot records the template hash in every model fingerprint.
  2. Diff against the claimed parent before deploying a derivative. This is ingot scan: template drift vs. parent is a first-class finding, including for quantized re-releases ("same model, smaller" is a claim — check it).
  3. Restore the parent template when drift is unintentional. Template drift is the one finding class that is mechanically fixable: ingot patch emits a pinned, content-hashed merge op restoring the parent's template. (Intentional drift — an instruct tune of a base model — is detected and left alone.)
  4. Gate CI on template-hash change. A template edit in a model repo should be a reviewed diff, exactly like a code change, because it is one.
  5. Golden-prompt the contract, not just the weights. A short deterministic suite — tool call with nested args, zero-arg tool call, thinking-mode round-trip, system-prompt echo — catches template/parser disagreement that no static diff can.

Limitations

Sources

Census method and raw evidence: REPORT.md (stage A) and out/FINDINGS.md; production census results are public per-model at ingot.tools. External incidents are linked inline; deployment-quirk survey in research/open-model-deployment-quirks.md.