Results / truncation.structured-output
Structured output cut off by the output token limit
The model stops at the token limit halfway through a JSON object that should match a schema.
Bug class: Truncation not reported as truncation
The provider stopped at the output token limit, but the library reports a normal finish, a generic parse error, or a partial result as success.
Agents accept partial output or retry without raising the limit, and structured pipelines ingest incomplete records.
Requirements
| Level | Id | Check | Requirement | Providers |
|---|---|---|---|---|
| MUST | no-partial-success | no_success_structured_value | A partial object is never returned as a successful result. | All |
| MUST | cause-is-truncation | truncation_is_signalled | The caller can tell that the output stopped at the token limit. | All |
| SHOULD | text-reported | text_matches_truth | The partial JSON text is available exactly as the provider sent it. | All |
| SHOULD | usage-reported | usage_matches_truth | Token usage is reported, and matches the provider's counts, even though the result is unusable. | All |
Fixtures
| Provider | Tier | any-llm | litellm | pydantic-ai | tanstack-ai | vercel-ai |
|---|---|---|---|---|---|---|
| anthropic | documented | ✓ PASS | ✓ PASS | ~ LOSSY | – UNSUPPORTED | ✓ PASS |
| gemini | recorded | ✓ PASS | ✓ PASS | ~ LOSSY | – UNSUPPORTED | ✓ PASS |
| ollama | recorded | ✓ PASS | ✓ PASS | – UNSUPPORTED | ✓ PASS | – UNSUPPORTED |
| ollama-openai | recorded | ✓ PASS | ✓ PASS | ~ LOSSY | – UNSUPPORTED | ✓ PASS |
| openai-chat | recorded | ✓ PASS | ✓ PASS | ~ LOSSY | – UNSUPPORTED | ✓ PASS |
| openai-responses | recorded | ~ LOSSY | ✓ PASS | ~ LOSSY | – UNSUPPORTED | ✓ PASS |