Your inference server silently ate the agent's turn

The model produced the right answer. The serving layer's parser dropped it, misrouted it, or corrupted it into garbage — and returned HTTP 200. We reproduced four such failures against the real, current vLLM parser code.

Published 2026-08-25. Findings come from executing the installed upstream vllm package (versions 0.26.0 and 0.27.1) against the exact reproducer idioms from open vLLM issues — CPU imports, no GPU, no model weights — plus renders of the real community chat templates the issues name. Every referenced issue was open as of retrieval on 2026-08-25. Probe scripts and raw transcripts are public in the ingot repository. This is not a safety certification of any model or serving framework.

---

Executive summary

Every report we have published so far is about the model artifact — its weights, its template, its stop tokens. This one is about the layer above it: the inference server that turns a model's raw text into the structured tool_calls and reasoning_content fields your agent reads. That layer has its own parsers, and they have their own bugs — bugs that are invisible in exactly the way that matters most, because the model did nothing wrong and the request returned success.

We reproduced four such failures against the real, installed vLLM parser code, not a simulation:

• The tool call silently vanishes. A correctly formatted Gemma4 tool call in one documented opener form parses to tools_called=False, an empty call list, and content=None. The agent sees no tool call, no content, and no error.
• The tool call is corrupted into garbage — and eats the next one. A parenthesized Gemma4 call produces a function name of terminal(command:<|"|>ls -a<|"|>)<tool_call|> with empty arguments, and a following valid call gets absorbed into that same garbage name instead of parsing on its own.
• The whole answer is filed as private reasoning. A compliant Qwen3 model answering in plain text has its entire answer routed into reasoning_content with content: null, because the parser's assumed thinking-state disagrees with what the chat template actually rendered.
• The parser decides reasoning ended before it began. A MiniMax M3 parser reads balanced think-tag examples in the prompt and concludes reasoning has already ended, engaging output grammar at the wrong time.

All four share one signature: HTTP-shaped success carrying wrong or missing structured output. No exception, no 4xx, no warning — just a null field, an empty array, or a corrupted string that a downstream agent treats as ground truth. These are defects in the serving stack, a surface no artifact scanner inspects, and every one is version- and lane-aware detectable with a release gate.

The parser is a trust boundary nobody tests

When you deploy a chat model behind an OpenAI-compatible server, the model emits raw text with special markers — <tool_call>, <think>, terminators. The server's parsers convert that text into the JSON your application consumes: message.tool_calls, message.reasoning_content, message.content. Your agent never sees the raw text. It trusts the parser.

That trust is the vulnerability. A parser is a small state machine, and when its grammar doesn't cover an input the model legitimately produces — a different-but-valid opener, a parenthesis, a template that closed the think block through an unexpected switch — it doesn't crash. It produces a confident, well-formed, wrong result. The response validates. The status code is 200. And the agent acts on structured data that does not match what the model said.

Below, both parser layers — tool-call and reasoning — fail this way, and both are live on current vLLM main.

Tool-call parsing: the turn that vanishes, and the one that corrupts

We executed vLLM 0.27.1's Gemma4 tool parser directly against the reproducer inputs from three open upstream issues. The results:

Silent drop (issue #53431). A bare tool-call opener of the form <|tool_call>:name{...} — a documented, valid form — hits a state machine with no transition for the bare :. The TOOL_PREAMBLE state has no way to emit the span, so it is emitted nowhere:

tools_called=False   tool_calls=[]   content=None

The model called a tool. The agent receives no tool call, no content, and no error. The reporter measured 386 failed turns over 21 days (~0.4%) in production; greedy decoding reproduces it every time.

Garbage name that absorbs the next call (issue #53642). A parenthesized call — call:name(...) — hits TOOL_NAME, a state with exactly one outgoing transition (for {). On (, the machine stays in TOOL_NAME and appends everything that follows into the function name:

name = terminal(command:<|"|>ls -a<|"|>)<tool_call|>   args = {}

Worse, when a correct call follows the broken one, both collapse into a single call: the garbage name runs on to include the second call's call:terminal, and the arguments carry only the second call's payload ({"command": "pwd"}). One malformed call silently swallows the valid call after it.

Streaming reasoning misclassification (issue #53246). The Kimi K3 streaming reasoning path, when the model omits an internal think-transition marker, classifies a complete tools block as reasoning. This one is lane- and version-specific: on our pinned 0.27.1 the non-streaming path appears repaired (it forwards the text downstream rather than reproducing the reported loss), while the streaming path still loses the turn. We report that split honestly — the bug is real on the streaming lane and already fixed on the non-streaming lane in this version.

These are not fixed on main. We fetched the current gemma4 parser source from vLLM main on 2026-08-25: TOOL_PREAMBLE still has no transition for a bare :, TOOL_NAME still has exactly one outgoing transition, and there is no parenthesis, escape, or backtracking handling anywhere in the file. The defect spans at least 0.26.0 through 0.27.1 and is live on current main — it is not a fresh regression that a version bump clears.

Reasoning parsing: the answer filed as a secret, both directions

The reasoning parser decides which part of a model's output is private chain-of-thought (reasoning_content) and which is the answer (content). It can fail in both directions, and we confirmed both against installed vLLM 0.26.0 and 0.27.1 with identical results on each.

Polarity A — the answer becomes reasoning (Qwen3, issue #53284). Qwen3Parser reads only the enable_thinking chat-template kwarg (default True) and never inspects the rendered prompt to check the actual think-state. We verified from installed source that its adjust_initial_state_from_prompt hook is an inherited no-op, that parse_delta accepts prompt_token_ids and provably ignores it (identical output with closed-think IDs and with empty IDs), and that the non-streaming path has no prompt parameter at all. The consumer is live in the serving layer (serving.py:339).

The consequence: when a chat template disables thinking by rendering a closed think block — through reasoning_effort=none, auto_disable_thinking_with_tools, or a custom community template — but the request never sets enable_thinking=false, a compliant model answers in plain text and the parser routes the entire answer into reasoning_content, leaving content: null. An agent reading message.content gets nothing, and retries or hallucinates.

We confirmed this with the real froggeric/Qwen-Fixed-Chat-Templates template (SHA-256 pinned), rendered with the documented triggers: it renders <think>\n\n</think>\n\n, and the real Qwen3Parser misroutes the plain answer into reasoning with content=None. The top-level OpenAI reasoning_effort parameter is safe — vLLM maps it correctly — so the hazard lives specifically in chat_template_kwargs and custom templates, which is exactly where teams running community templates operate.

Polarity B — reasoning "ended" before it started (MiniMax M3, issue #46042). MiniMaxM3ReasoningParser.is_reasoning_end does a naive backward scan (last close marker after last start marker means "ended"), while the same class's own count_reasoning_tokens uses correct depth counting. The two disagree. When the rendered prompt contains a balanced think-tag example — as MiniMax M3's own stock template does, instructing the model to wrap reasoning in <mm:think></mm:think> — the backward scan reads the example as proof that reasoning has already ended and can engage output grammar at the wrong time. The is_reasoning_end code is byte-identical on 0.26.0, 0.27.1, and current main: the defect is live upstream.

Why this surface is different

Everything Ingot has reported until now is a property of a file you can download and inspect: a weight, a template, a config key. This is a property of the running server — the parser version, the streaming vs. non-streaming lane, the interaction between a chat template's rendered state and the parser's assumed state. You cannot find it by scanning a repository, because the model repository is innocent. The model produces correct output; the server mistranslates it.

That makes it invisible to every existing defense in a way even template drift is not. A template diff catches a changed file. Nothing catches a parser that returns content: null for a valid answer, because there is no changed file — only a version of vLLM whose parser grammar doesn't cover an input the model legitimately emits.

What it costs

• Dropped actions. A vanished tool call means the agent's step simply doesn't happen — no error to catch, no retry triggered by a failure it never sees.
• Corrupted actions. A garbage function name that absorbs the next call can turn two intended operations into one malformed one, with arguments from the wrong call.
• Silent retries and hallucination. An answer filed as reasoning leaves content: null; an agent reading content gets nothing and either loops or fabricates.
• Undetectable in aggregate. Every one of these returns HTTP 200. Standard monitoring — error rates, status codes, latency — shows a healthy service. The reporter of #53431 needed 21 days and a manual audit to find a 0.4% turn-loss rate.

How to catch it

The check is a version- and lane-aware release gate, run against the exact vLLM version and configuration a deployment uses — which is precisely how we produced this report:

1. Execute the installed parser against a battery of known-hazard inputs — the documented opener forms, parenthesized calls, a valid call following a malformed one, closed-think and open-think prompts, balanced-example prompts — and assert the structured output matches the model's actual intent. No GPU or weights required; the parsers run on a CPU import in seconds.
2. Test both lanes. Streaming and non-streaming parsers diverge (the Kimi K3 and Qwen3 cases both differ by lane). A gate that tests only one lane misses half the surface.
3. Pin the version. These defects are version-scoped: some are fixed on one lane and live on another within the same release. The gate must record the vLLM version it ran against and re-run on every upgrade, because a bump can fix one row and regress another.
4. Render real templates. The most dangerous Qwen3 case only appears when a specific community template renders a closed think block. The gate should render the actual templates a deployment serves, not just synthetic inputs.

Limitations

• Reproductions are at the parser-code level: we import and run the real vLLM package against reproducer strings. We did not run a live server or measure production emission rates; the reporters' rates (0.4% turn loss over 21 days; 0–53% of parenthesized calls per boot on some quantized checkpoints) are their measurements, not ours.
• The Kimi K3 finding is version- and lane-scoped: the streaming lane reproduces on 0.27.1; the non-streaming lane appears repaired there; 0.25/0.26 non-streaming behavior was not retested, and the module is absent in 0.26.0.
• We reproduced the mechanism and the misrouted output, not the downstream business impact — how often a real agent loop hits these inputs depends on the model, the template, and the traffic.
• The upstream issues were open at retrieval on 2026-08-25 (#53431, #53642, #53246, #53284, #46042); a proposed fix for #53284 (PR #53302) was open and unmerged. Versions move; the gate approach is what generalizes, not the specific version rows.
• Named parsers and issues are cited for a correctness defect in a specific version, not as a judgment of vLLM overall — vLLM is the framework precisely because it is the one whose parser code is public and reproducible.

Sources

Probe scripts that import and execute the real vLLM parsers, the version sweep, and raw transcripts are in research/experiments/2026-08-25-tool-parser-fsm/ (tool-call parsing) and research/experiments/2026-08-25-reasoning-parser-mismatch/ (reasoning parsing, both polarities, with pinned real-template renders). Upstream issues: vLLM #53431, #53642, #53246, #53284, #46042. These parsers are the serving-layer companion to the artifact-level failures in REPORT-STOP-TOKEN.md and REPORT-TEMPLATE-DRIFT.md.