Repository navigation
[Server] Validate tool output against outputSchema - #515
chr-hertel merged 2 commits into
Conversation
Sorry, something went wrong.
| return null; | ||
| } | ||
|
|
||
| $validationErrors = $this->schemaValidator->validateAgainstJsonSchema($result->structuredContent, $tool->outputSchema); |
There was a problem hiding this comment.
$result->structuredContent could also be an empty list - which is valid as result, but validateAgainstJsonSchema turns that into new \stdClass()
Sorry, something went wrong.
There was a problem hiding this comment.
or does that work against an array schema still 🤔 shouldn't right?
Sorry, something went wrong.
There was a problem hiding this comment.
good catch, the coercion now only applies to arguments (always an object), [] is validated as an array. While at it an empty object result was sent as [] on the wire, it's now kept as {}.
Sorry, something went wrong.
chr-hertel
left a comment
There was a problem hiding this comment.
Thanks @soyuka!
Sorry, something went wrong.
The specification requires a server to produce structured results that conform to a declared outputSchema (2025-11-25 server/tools.mdx:340), but the SDK only validated arguments against inputSchema. A tool could ship non-conforming structuredContent and strict clients would reject the call with no diagnostic on the server side. A mismatch is now answered with a CallToolResult carrying isError: true, so the model reads the reason and can fall back to content, as the TypeScript, Python and Java SDKs do. Validation is skipped when the tool declares no outputSchema, when the result carries no structuredContent (already warned about), and when the result is already an error.
An empty PHP array was coerced to an object before schema validation,
so `[]` failed an array outputSchema although it is sent as `[]`. The
coercion now only applies to tool arguments, which are always an
object. An empty object result is kept as stdClass so it is sent as
`{}` rather than `[]`.
Problem
The specification states that a server must produce structured results that conform to a declared outputSchema:
The wording is the same in 2025-11-25, 2026-07-28 and the draft.
CallToolHandler validated the arguments against inputSchema, then sent the result without checking it against outputSchema. A tool could return non-conforming structuredContent. A strict client then rejected the call, and the server logged nothing about why.
Change
CallToolHandler::validateStructuredContent() runs on the result just before it is returned. It covers a value the SDK wrapped and a CallToolResult the tool built itself.
On a mismatch the handler logs at error level and answers with a tool execution error:
{ "content": [ { "type": "text", "text": "Invalid structured output for tool 'get_weather': Missing required properties: `temperature`." } ], "isError": true }A tool execution error keeps the reason visible to the model, which can then fall back to content. A protocol error would hide it.
The check is skipped in three cases:
The error summary code that built the inputSchema message is now summarizeValidationErrors(), shared by both paths instead of duplicated.
How the other SDKs behave
TypeScript and Java both skip the check on an isError result, and both validate a result the tool built itself. This PR follows them.
SEP-2140 proposes the same behavior in item 4: report output validation failures as tool execution errors. Its tool-resolution half already landed in this SDK. Note that the output-validation wording is not yet in the draft specification, so this PR follows the SEP as a precedent, not as binding text.
Compatibility
A server whose tool emits non-conforming structuredContent today gets a successful result. After this change it gets an error result. The CHANGELOG entry is marked [BC Break] for that reason. Tell me if you want the tag dropped, because the earlier behavior violated the specification.
Tests
Six tests in tests/Unit/Server/Handler/Request/CallToolHandlerTest.php, each written to fail first:
make cs, make phpstan and the touched test files pass. tests/Inspector/Stdio/StdioEnvVariablesTest.php passes too, because examples/server/env-variables/EnvToolHandler.php is the one fixture that declares an outputSchema end to end.