FacebookAI/roberta-large warn
chat template: not in config · view on Hugging Face ↗
Ingot findings
Static battery clean: safetensors weights, license declared, no template/tokenizer drift detected. Deep battery not yet run. Weights battery: embedding-norm scan over 50265 tokens (F32, 1024-dim) found 175 undertrained candidates, 97 plain-ASCII. Scanned 2026-08-20 (published from a community scan).
medium Undertrained (glitch) token surface in vocabulary
Embedding-norm scan flagged 175 undertrained tokens (norm < 0.3× the vocabulary median of 4.489), including 97 plain-ASCII strings that can appear in ordinary input as identifiers — e.g. "PsyNetMessage", "FactoryReloaded", "TheNitrome", "Adinida", "channelAvailability", "NetMessage", "attRot", "Orderable". In models where this class was tested behaviorally, such tokens silently rewrote user input into confident, schema-valid, wrong output. These are candidates from the weights alone; behavioral confirmation requires the GPU deep battery.
How to fixruntime guardweight-level
Keep the affected token strings out of the model's input — the scan-derived runtime guard carries this model's exact blocklist.
- Fetch this model's guard artifact (`/api/v1/guard/<owner>/<model>`): the confirmed corrupting tokens and the low-norm candidate list, derived from the published scan.
- Screen inbound text with it (the `@ingotai/guard` package is a reference implementation) and route flagged records to a different model or human review — verbatim-copy tasks on flagged strings are the failure mode.
- The underlying cause is undertrained embeddings in the weights; a true fix is weight-level (continued pretraining on the affected tokens) — that is not a patch, it's a training job.
Remediation guidance addresses the documented findings only. It is evidence-driven repair, not a safety certification of the model.
Fingerprint
The durable weights-and-metadata profile of this model, rebuilt on every scan and deep-battery run. Updated 2026-08-20.
| architecture | roberta · 24 layers · 1024-dim |
| parameters | 355.4M |
| vocabulary | 50,265 tokens |
| license | mit |
| serialization | safetensors pickle |
| chat template | none |
| glitch-token surface | 175 undertrained candidates, 97 plain-ASCII |
Full fingerprint
| architectures | RobertaForMaskedLM |
| library | transformers |
| pipeline | fill-mask |
| repo files | 12 — pickle: pytorch_model.bin |
| revision | 722cf37b1afa |
| HF snapshot | 11.1M downloads · 319 likes · updated 2024-02-19 · captured 2026-08-20 |
| embedding tensor | roberta.embeddings.word_embeddings.weight · F32 · 50,265×1024 |
| embedding norms | median 4.4894 · mean 4.3585 |
| lineage check | no claimed base model |
| glitch-token samples | "PsyNetMessage", "FactoryReloaded", "TheNitrome", "Adinida", "channelAvailability", "NetMessage", "attRot", "Orderable", "guiIcon", "isSpecialOrderable", "srfN", "guiActiveUn" |
Verdict badge
Ship the verdict in your README — it always shows the latest published analysis:
[](https://ingot.tools/models/FacebookAI/roberta-large)