~ LOSSY
any-llm 1.40.2
truncation.structured-output · openai-responses
Requirements
| Id | Level | Status | Evidence |
|---|---|---|---|
| no-partial-success | MUST | SATISFIED | structured.attempted=true; structured.returned_as_success=false |
| cause-is-truncation | MUST | NOT_OBSERVABLE | finish.value=null (absent); truth.finish="length"; error.category="validation"; structured.returned_as_success=false |
| text-reported | SHOULD | NOT_OBSERVABLE | text.value=null (absent); truth.text="{\"islands\":[{\"name\":\"Κρήτη\",\"sea\":\"Κρητικό Πέ" |
| usage-reported | SHOULD | NOT_OBSERVABLE | usage.input=null (absent); truth.usage.input=68; usage.output=null (absent); truth.usage.output=24 |
What the provider sent
- Finish
- length
- Raw finish
- max_output_tokens
- Text
- {"islands":[{"name":"Κρήτη","sea":"Κρητικό Πέ
- Tool calls
[]
- Usage
{ "alternatives": {}, "cached_input": 0, "input": 68, "output": 24, "raw": { "input_tokens": 68, "input_tokens_details": { "cache_write_tokens": 0, "cached_tokens": 0 }, "output_tokens": 24, "output_tokens_details": { "reasoning_tokens": 0 }, "total_tokens": 92 }, "reasoning": 0, "total": 92 }- Stream completed
null
- Error
null
Wire excerpt
status: 200
content-type: application/json
body (json):
{
"id": "resp_04ca3cce672b80e9006acbfbfeef9c87d1b371f55d49b29248",
"object": "response",
"created_at": 1791753214,
"status": "incomplete",
"access_programs": null,
"background": false,
"billing": {
"payer": "developer"
},
"completed_at": null,
"error": null,
"frequency_penalty": 0.0,
"incomplete_details": {
"reason": "max_output_tokens"
},
"instructions": null,
"max_output_tokens": 24,
"max_tool_calls": null,
"model": "gpt-6-luna",
"moderation": null,
"output": [
{
"id": "msg_04ca3cce672b80e9006acbfbffd3a087d1882b5ea5d91c9c0b",
"type": "message",
"status": "incomplete",
"content": [
{
"type": "output_text",
"annotations": [],
"logprobs": [],
"text": "{\"islands\":[{\"name\":\"Κρήτη\",\"sea\":\"Κρητικό Πέ"
}
],
"phase": "final_answer",
"role": "assistant"
}
],
"parallel_tool_calls": true,
"presence_penalty": 0.0,
"previous_response_id": null,
"prompt_cache_key": null,
"prompt_cache_retention": "24h",
"reasoning": {
"context": "all_turns",
"effort": "none",
"mode": "standard",
"summary": null
},
"safety_identifier": null,
"service_tier": "default",
"store": true,
"temperature": 1.0,
"text": {
"format": {
"type": "json_schema",
"description": null,
"name": "output",
"schema": {
"type": "object",
"properties": {
"islands": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"sea": {
"type": "string"
}
},
"required": [
"name",
"sea"
],
"additionalProperties": false
}
}
},
"required": [
"islands"
],
"additionalProperties": false
},
"strict": true
},
"verbosity": "medium"
},
"tool_choice": "auto",
"tool_usage": {
"image_gen": {
"input_tokens": 0,
"input_tokens_details": {
"image_tokens": 0,
"text_tokens": 0
},
"output_tokens": 0,
"output_tokens_details": {
"image_tokens": 0,
"text_tokens": 0
},
"total_tokens": 0
},
"web_search": {
"num_requests": 0
}
},
"tools": [],
"top_logprobs": 0,
"top_p": 0.98,
"truncation": "disabled",
"usage": {
"input_tokens": 68,
"input_tokens_details": {
"cache_write_tokens": 0,
"cached_tokens": 0
},
"output_tokens": 24,
"output_tokens_details": {
"reasoning_tokens": 0
},
"total_tokens": 92
},
"user": null,
"metadata": {}
}What the library reported
- Outcome
- raised
- Finish
{ "raw": null, "source": null, "surface": "absent", "value": null }- Text
- Value
null
- Surface
- absent
- Tool calls
[]
- Usage
{ "cached_input": null, "input": null, "output": null, "reasoning": null, "surface": { "cached_input": "absent", "input": "absent", "output": "absent", "reasoning": "absent", "total": "absent" }, "total": null }- Stream
{ "chunks_seen": null, "completed": null, "used": false }- Error
- Type
- pydantic_core._pydantic_core.ValidationError
- Message
- 1 validation error for output Invalid JSON: EOF while parsing a string at line 1 column 59 [type=json_invalid, input_value='{"islands":[{"name":"Κ...a":"Κρητικό Πέ', input_type=str] For further information visit https://errors.pydantic.dev/2.14/v/json_invalid
- Category
- validation
- Retry after
null
- Stack top
- pydantic/main.py:847
- Unsupported reason
null
Full observation JSON
{
"elapsed_ms": 318.62801872193813,
"error": {
"category": "validation",
"message": "1 validation error for output\n Invalid JSON: EOF while parsing a string at line 1 column 59 [type=json_invalid, input_value='{\"islands\":[{\"name\":\"Κ...a\":\"Κρητικό Πέ', input_type=str]\n For further information visit https://errors.pydantic.dev/2.14/v/json_invalid",
"retry_after": null,
"stack_top": "pydantic/main.py:847",
"type": "pydantic_core._pydantic_core.ValidationError"
},
"finish": {
"raw": null,
"source": null,
"surface": "absent",
"value": null
},
"format": "wiretruth.observation/v1",
"library": {
"api": "any_llm.responses",
"language": "python",
"name": "any-llm",
"version": "1.40.2"
},
"notes": [],
"outcome": "raised",
"stream": {
"chunks_seen": null,
"completed": null,
"used": false
},
"structured": {
"attempted": true,
"returned_as_success": false,
"value": null
},
"text": {
"surface": "absent",
"value": null
},
"tool_calls": [],
"unsupported_reason": null,
"usage": {
"cached_input": null,
"input": null,
"output": null,
"reasoning": null,
"surface": {
"cached_input": "absent",
"input": "absent",
"output": "absent",
"reasoning": "absent",
"total": "absent"
},
"total": null
}
}Fixture
recorded openai-responses/truncation.structured-output
| Provenance | Value |
|---|---|
| Recorded at | 2026-10-11T21:13:34Z |
| Requested model | gpt-6-luna |
| Returned model | gpt-6-luna |
| CI run | https://github.com/kartsan03/wiretruth/actions/runs/38175457862 |
Reproduce
wiretruth serve openai-responses/truncation.structured-outputPlain-library snippet from the shim manifest.
any_llm.responses(model=model, provider='openai', api_base=WIRETRUTH_BASE_URL_OPENAI_RESPONSES, api_key=WIRETRUTH_API_KEY)Upstream
No matching upstream report.
History
| Version | Grade |
|---|---|
| 1.40.2 | ~ LOSSY |
| 1.38.0 | ~ LOSSY |
| 1.33.0 | ~ LOSSY |