The GGUF you downloaded froze someone else's bug

Quantized conversions copy a model's configuration at a moment in time — including its bugs — and upstream fixes never reach the copy. We found the pattern in 43% of popular GGUF repositories, including launch bugs that were fixed a year ago and are still downloadable today.

Published 2026-08-25. Findings come from a streaming metadata sweep of the most-downloaded GGUF repositories on Hugging Face (file headers read over HTTP range requests — no multi-gigabyte downloads), a config-drift comparison of derivative/base model pairs, and one controlled behavioral experiment in llama.cpp. Probe scripts and raw artifacts are public in the ingot repository. This is not a safety certification of any model.

---

Executive summary

A GGUF file is a snapshot. When a quantizer converts a model for llama.cpp and Ollama, it copies the source repository's chat template, its stop-token configuration, and its tokenizer metadata into the .gguf header at conversion time — and then that copy never updates. If the source model had a bug the day it was converted, the GGUF still has that bug today, even after the original authors fixed it upstream. The weights are faithfully quantized; the configuration is faithfully frozen, mistakes and all.

We measured how often the frozen copy has already diverged from the model it claims to be:

• 40 of 92 parseable popular GGUF repositories (43%) embed a chat template, stop-token set, or tokenizer setting that differs from their own source repository.
• 140 of 412 derivative/base model pairs (34%) dropped at least one of the base model's end-of-sequence token IDs — sometimes harmlessly, sometimes dropping the very token that stops generation.
• The drift includes resurrected launch bugs still downloadable today: one popular Phi-4 GGUF embeds both of the original Phi-4 launch bugs that were fixed upstream in January 2025.

The most important consequence is silent. In a controlled experiment we removed a single tokenizer metadata key from a known-good Qwen GGUF. llama.cpp loaded it without error — only a log warning most GUI users never see — and tokenization changed on a code-and-math probe, splitting merges that a reasoning model depends on. The file looked fine, loaded fine, and served wrong tokens.

Every one of these defects is a cross-check of files already in the repository against the artifact's own declared lineage. None is caught by any scanner attached to the Hub.

The frozen-snapshot problem

The standard llama.cpp workflow: a popular quantizer takes a source model from Hugging Face, runs convert_hf_to_gguf.py, and uploads the resulting .gguf. That conversion reads the source repository's tokenizer_config.json, config.json, generation_config.json, and chat template, and bakes their contents into the GGUF's metadata block. From that point the GGUF is self-contained — llama.cpp reads everything it needs from the file header.

Self-contained also means severed. When the source repository later fixes its chat template, corrects its stop-token set, or updates its tokenizer, the GGUF does not follow. There is no link back. A user who downloads the GGUF a year later gets the configuration as it stood on conversion day.

This is the same heritability we documented for chat-template drift and stop-token runaway generation: quantizers copy the artifact, bug included, and every weight checksum still passes. This report measures how widespread the frozen divergence is and shows that even a first-party, well-intentioned conversion carries it.

43% of popular GGUFs have already drifted

We read the metadata header of 92 parseable popular GGUF repositories over HTTP range requests — roughly 4 MB per file, no full download — and compared the embedded template, EOS set, and pre-tokenizer against the current source repository each was converted from. 40 (43%) differ. Ordered by severity:

Resurrected launch bugs. MaziyarPanahi/phi-4-GGUF, from a popular quantizer and currently downloadable, embeds both of the original Phi-4 launch bugs that unsloth fixed upstream in January 2025: its stop token is <|endoftext|> (100257) while its template ends every turn with <|im_end|>, and it ships the pre-fix template that force-appends <|im_start|>assistant<|im_sep|> after every user message. We verified the template diff against microsoft/phi-4's current template. The upstream fix exists; this conversion predates it and will carry the bug until it is re-uploaded. The same class appears in MaziyarPanahi/Yi-Coder-1.5B-Chat-GGUF and -9B-Chat-GGUF (EOS 2 vs. the source's 7) and in MiniCPM5 thinking GGUFs (<|im_end|> template against a </s> stop token).

Degraded tokenization from a missing key. MaziyarPanahi/QwQ-32B-GGUF and MaziyarPanahi/Llama-3-8B-Instruct-64k-GGUF are BPE-tokenizer GGUFs with no tokenizer.ggml.pre key. When that key is absent, llama.cpp silently falls back to a default pre-tokenizer — the exact signature of the Llama-3 tokenization fiasco. QwQ is a reasoning model, where math and code tokenization damage is maximally harmful. We confirmed the behavioral impact directly (below).

Silently lost tool calling. Several Mistral GGUFs embed legacy [INST] templates of 415–1058 characters where the current source ships 1597–3959-character templates with tool-calling support. A llama.cpp user of the older conversion loses tool calling with no indication anything changed. OBLITERATUS/Qwen3.8-27B-OBLITERATED embeds a 506-character template against an 8952-character source.

Even first-party GGUFs drift. Qwen's own official Qwen/Qwen3-*-GGUF repositories embed an older template variant than Qwen's own source repositories — the GGUF version lacks the message.content is string list-content handling and uses different tool-response detection. Mostly benign, but it shows the freeze is structural, not a mark of a careless uploader: even the model's authors ship a snapshot that has since moved on.

The one-key defect, confirmed harmful

The missing tokenizer.ggml.pre case above is the kind of drift that sounds harmless — a metadata key, not a weight. We tested whether it actually changes behavior, without downloading either flagged model.

We took Qwen's official Qwen2.5-1.5B Q4_K_M GGUF and produced a byte-identical copy with only the tokenizer.ggml.pre key removed (metadata surgery that preserves file alignment), then loaded both in llama.cpp:

• The modified file loaded and served normally. The only signal was a log line — missing pre-tokenizer type, using: 'default' and GENERATION QUALITY WILL BE DEGRADED! — which Ollama and LM Studio GUI users never see.
• On a math-and-code probe string, tokenization diverged from 35 tokens to 37: .g split into . + g, +y into + + y. This is precisely the degradation class of the Llama-3 BPE fiasco — worst on numbers and code, and worst of all in a reasoning model like QwQ.

The lesson generalizes: the file passes every check a user can casually run. It loads, it responds, the weights are intact. Only a tokenization diff against the source — or noticing a warning line in a log — reveals that it is quietly serving different tokens than the model it claims to be.

The stop-token set shrinks across lineages too

Beyond GGUFs, we compared 412 derivative/base model pairs (full configs on both sides) for configuration drift against the claimed base:

• EOS-set shrinkage: 140 of 412 pairs (34%) dropped at least one of the base model's end-of-sequence token IDs. Some cases are benign (dropping a padding <|endoftext|>); some are critical (dropping the template's terminator, which produces the runaway generation documented in the stop-token report). The severity depends entirely on which ID was lost — an absolute count tells you nothing.
• 137 pairs dropped the base model's generation_config.json entirely — the exact configuration-loss mode behind the original Llama-3 stop-token failure.
• Rope-scaling and position-embedding drift appear in a further set of pairs, though many are legitimate retrains (e.g. Qwen/Qwen2.5-Math-1.5B changing rope_theta from its tagged base is an intended math specialization, not a defect).

Why absolute rules don't work — and lineage rules do

We also ran a template security lint (looking for injection, exfiltration URLs, dunder traversal) across 193 templates, and a double-BOS sweep across 76 top models. Both came back clean: 0 genuinely malicious templates in the top set, and 0 double-BOS defects — the 2024 bug class where a template emits {{bos_token}} while the tokenizer also auto-adds it appears cleaned up in currently-popular models.

Those negatives are the point. Every absolute heuristic we tried over-flagged badly: naive template-injection rules flagged 122 of 193 templates (worthless), and the naive "missing BOS" rule flagged 21 of 76 models that intentionally start without BOS (gpt-oss, ChatML families, granite, Nemotron). Meanwhile every check that survived validation is relational or exact: compare the artifact against its declared base, its source repository, or its own other files, and flag only a concrete disagreement — a lost terminator, a template diff, a missing required key, an exactly-zero embedding row.

This is the same reason Ingot's original template-drift scan works. The Hub has no absolute invariants — models legitimately differ in every dimension — only lineage invariants. A scanner that asks "does this artifact match the model it claims to be a copy of?" finds real bugs. A scanner that asks "does this artifact match a fixed idea of a correct model?" drowns in false positives.

What it costs

• A downloaded fix that was never applied. Users choose GGUFs specifically to run a known, red-teamed model on their own hardware. When the conversion froze a since-fixed bug, they inherit a defect the model's authors already repaired — silently, because the weights match and the file loads.
• Degraded reasoning and code output. The missing-pre-tokenizer case damages exactly the tokens (numbers, code, math) that reasoning models depend on, with no error and no crash.
• Broken tool calling. A conversion that froze a pre-tool-calling template removes a capability the source model advertises, so an agent built against the model card fails against the artifact.
• An audit trail that lies. The GGUF passes checksum and load-time validation. The evidence that the model was reviewed applies to the source repository as it stands today, not to the frozen snapshot in the file.

How to catch it

Every input is already in the repository, and every check is a comparison against declared lineage:

1. Diff the GGUF's embedded template, EOS set, and pre-tokenizer against the current source repository it was converted from. Flag a lost terminator, a template that lost tool-calling support, or a shrunken stop set.
2. Flag a BPE GGUF (tokenizer.ggml.model == "gpt2") missing tokenizer.ggml.pre as high severity — it triggers the silent default-pre-tokenizer fallback. Optional confirmation tier: diff llama-tokenize against the HF tokenizer on a probe corpus (CPU, seconds).
3. Compare a derivative's EOS set against its claimed base, grading by which ID was lost rather than by count. A lost template terminator is high severity; a dropped padding token is not.
4. Flag a dropped generation_config.json when the base has one.

The remediation for the metadata cases is a one-file, in-place edit with gguf-py — no requantization required. The missing pre-tokenizer key, a frozen template, and a shrunken EOS set are all editable in the GGUF header directly, of the same shape as the ingot patch fix already emitted for template drift.

Limitations

• The GGUF sweep covers 92 parseable popular repositories; long-tail prevalence is unknown, and repositories whose headers did not parse were excluded rather than counted.
• Exact-match template diffing over-reports: some differences are minja-compatibility rewrites that don't change behavior. Separating those from real behavioral changes needs a semantic render-compare layer on canonical conversations; the 43% figure counts any embedded-metadata difference, and the severity grading (not the raw count) is where the judgment lives.
• The missing-pre-tokenizer behavioral confirmation is one model (Qwen2.5-1.5B) on one probe string in one llama.cpp version. It demonstrates the mechanism and its silence; it does not estimate how much output degrades across models or tasks.
• Named repositories are flagged for a configuration inconsistency against their own lineage, not for malice or poor quality. The fix in every case is a small metadata edit.
• Tensor-name checks need an architecture-aware map: DeepSeek-V4 names its output head head.weight, which a naive check misreads.

Sources

Probe scripts, the streaming GGUF parser, raw sweep outputs (gguf_sweep_report.txt, base_drift_report.txt, template_lint_report.txt), and the config-drift comparison are in research/experiments/2026-08-25-gguf-metadata-drift/. The missing-pre-tokenizer behavioral confirmation and the double-BOS/missing-BOS sweep are in research/experiments/2026-08-25-pretok-bos-behavioral/. Companion reports: REPORT-TEMPLATE-DRIFT.md (templates that change without a weight moving) and REPORT-STOP-TOKEN.md (the stop-token incoherence this drift inherits and freezes).