Results / tools.parallel-calls
Parallel tool calls for two cities
The model returns one weather tool call for each of two cities.
Bug class: Tool calls lost, merged or returned with broken arguments
The library drops or merges tool calls, mixes up their ids, or returns a call as complete while its arguments are cut off or repaired.
Tools run with wrong or incomplete arguments, or don't run at all.
Requirements
| Level | Id | Check | Requirement | Providers |
|---|---|---|---|---|
| MUST | calls-reported | tool_calls_match_truth | Both tool calls are reported with the ids, names and arguments the provider sent. | All |
Fixtures
| Provider | Tier | any-llm | litellm | pydantic-ai | tanstack-ai | vercel-ai |
|---|---|---|---|---|---|---|
| anthropic | documented | ✓ PASS | ✓ PASS | ✓ PASS | – UNSUPPORTED | ✓ PASS |
| gemini | recorded | ✓ PASS | ✗ WRONG | ✓ PASS | – UNSUPPORTED | ✓ PASS |
| ollama | recorded | ✗ WRONG | ✓ PASS | – UNSUPPORTED | – UNSUPPORTED | – UNSUPPORTED |
| ollama-openai | recorded | ✓ PASS | ✓ PASS | ✓ PASS | – UNSUPPORTED | ✓ PASS |
| openai-chat | recorded | ✓ PASS | ✓ PASS | ✓ PASS | – UNSUPPORTED | ✓ PASS |
| openai-responses | recorded | ✓ PASS | ✓ PASS | ✓ PASS | – UNSUPPORTED | ✓ PASS |