Sign in

What Actually Changes in "Uncensored" Open Models?

A forensic diff of popular Hugging Face derivatives — provenance, weights, and behavior. Viability spike for an open-model forensics scanner.

Status: research spike, 2026-08-17/18. Numbers are spike-sized (see Limitations) and should be scaled before publication. Not a safety certification of any model.

TL;DR

We scanned the most-downloaded community derivatives of three open base models (Qwen3-8B, Qwen3-4B, Qwen2.5-7B-Instruct) at three depths — repository metadata, weight structure, and behavior — and diffed each derivative against its claimed parent.

Three things stand out:

  1. Most derivatives silently change things they don't disclose. 56% ship a different chat template than their parent; 37% never mention it. Among pure quantization re-releases — where users assume identical behavior — 78% changed or dropped the chat template.
  2. Modification techniques have distinct, machine-checkable weight fingerprints. Two independently-produced "abliterated" models converged on the identical signature: rank-1 edits to only the attention-output and MLP-down projections. Full fine-tunes and light preference-tuning look completely different.
  3. You cannot infer behavior from how much the weights moved. The derivative with the smallest weight change we measured removed the most safety: harmful-request refusal fell from 100% to 5% with zero capability loss.

The takeaway for anyone running third-party weights: provenance claims and even weight-size scans are not enough. What a model does has to be measured directly and differentially against its parent.

Method

Three stages, each diffing derivative against claimed parent, cheapest first.

stage what cost
A — census repo metadata: files, configs, tokenizer, chat template, README claims, per-tensor shapes/dtypes no downloads (HF API + HTTP range reads)
B — weight diff ΔW per tensor: relative-L2, cosine, layer/module heatmap, SVD low-rank fingerprint streams weights; ~15 min for 5 pairs on a CPU VM
C — behavioral diff 90 identical prompts/model (refusal, capability, EN/ZH), deterministic classification one L4 GPU, <1 min/model

296 derivatives in stage A; 5 parent/derivative pairs in stage B; 2 pairs in stage C. All raw evidence (tensor stats, refusal classifications) is preserved for reproduction.

Stage A — the census (296 models)

signal rate
chat template differs from parent 166 / 296 (56%)
...undisclosed (README never mentions it) 110 / 296 (37%)
special/added tokens changed 161 / 296 (54%)
pickle-format files present (.bin/.pt/.pkl) 51 / 296 (17%)
pure quantization re-releases altering the template 25 / 32 (78%)

Chat-template drift is behavioral drift: the template controls the default system prompt, thinking-mode parsing, and tool-call format. A quantization that silently ships a different template can change model behavior even though the weights are "the same model."

Individually interesting cases surfaced by the census alone:

No template injection attacks (hidden instructions/URLs) appeared in this sample; the template edits were rebrands, system-prompt swaps, and tool-format changes. No pure sha-level re-uploads either.

Stage B — weight fingerprints (5 pairs)

For each pair we compute ΔW = W_derivative − W_parent per tensor, aggregate by layer and module, and take the singular values of the most-changed matrices.

derivative tensors changed where median rank@90% of ΔW reads as
mlabonne/Qwen3-4B-abliterated 72 / 398 only o_proj + mlp.down_proj 1 abliteration
huihui Qwen3-8B-abliterated-v2 72 / 399 only o_proj + mlp.down_proj 1 abliteration
Orion-zhen Qwen2.5-7B-Uncensored 196 / 339 broad, MLP-heavy 34 light SFT/DPO
t-tech/T-lite-it-2.1 (380k dl) 397 / 399 everything 82 honest full fine-tune
AvitoTech/avibe 396 / 399 + vocab swap everything 81 full FT + retokenization

The abliteration result is the striking one. Two people, different tools, different base models, converged on the same signature: leave 82% of tensors byte-identical, and edit only the attention-output and MLP-down projections, each edit almost exactly rank 1 (one dominant singular value ~500× larger than the next). That is the refusal-direction projection sitting directly in the weight delta — and it is a specific, checkable fingerprint, not a vibe.

This is the basis for a "modification recipe" database: a derivative's weight delta says a great deal about how it was made, independent of what its card claims.

Stage C — behavior (2 pairs)

Same 90 prompts to parent and derivative, each with its own chat template, greedy decoding. Refusal classified deterministically (marker list, English + Mandarin); we only record refuse-vs-comply, never the content.

pair harmful refusal (parent → deriv) capability benign over-refusal
Orion-zhen Qwen2.5-7B-Uncensored 100% → 5% 100% → 100% 0%
huihui Qwen3-8B-abliterated-v2 75% → 0% 95% → 100% 0%

The finding that ties it together: weight-subtlety ≠ behavioral-subtlety

Cross-reference stages B and C. Orion-zhen had the smallest weight change of anything we measured (relative-L2 0.0014, ~20× lighter than abliteration, embeddings and layernorms untouched). Yet it is behaviorally the most uncensored: harmful-request refusal collapses from 100% to 5% with no measurable capability loss. A change that barely registers on a "how much did the weights move" scan removes nearly all safety behavior.

Meanwhile huihui closes the loop the other way: its rank-1 abliteration fingerprint from stage B predicted the behavior we then measured — 75% → 0% refusal, capability intact.

Neither the provenance claim, nor the artifact scan, nor the weight-size alone would have told you what these models actually do. The differential — structure and behavior, against the parent — does.

Why this is a product, not just a blog post

Every column above is generated by the same differential primitive (derivative − parent) and is reproducible from preserved raw evidence. Stack the stages and you get a forensic report that answers a question people running open weights genuinely can't answer today: what is this checkpoint, how was it made, and what does it do differently from the thing it claims to be?

The accumulating asset is the fingerprint database — weight signatures, template diffs, refusal deltas keyed to known modification recipes — so the next unknown checkpoint can be placed against known ancestors and techniques automatically.

Limitations (read before quoting numbers)

Reproduce

census.py       # stage A: python3 census.py Qwen/Qwen3-8B ...
weightdiff.py   # stage B: python3 weightdiff.py <parent> <deriv> [--tag ..]
behavioral.py   # stage C (GPU): python3 behavioral.py --parent .. --deriv .. --tag ..
prompts.py      # stage C prompt sets

Findings detail: out/FINDINGS.md (A), out/PHASE-B-FINDINGS.md (B), out/PHASE-C-FINDINGS.md (C). GPU env combo that works: vllm==0.9.2 + transformers==4.53.2 + torch 2.7.0/cu126.