fix(llm): accept reasoning field in OpenAI-compatible streams - #35187
fix(llm): accept reasoning field in OpenAI-compatible streams#35187mickaeldamatha wants to merge 1 commit into
Conversation
Sorry, something went wrong.
Some OpenAI-compatible providers (Scaleway, vLLM/SGLang, OpenRouter, ...) stream chain-of-thought under a `reasoning` delta field instead of the `reasoning_content` field emitted by DeepSeek et al. The OpenAI Chat protocol only declared and read `reasoning_content`, so `reasoning` was stripped by the delta schema and the model's thinking was silently dropped for those providers. Declare `reasoning` on the delta schema and fall back to it when `reasoning_content` is absent, preferring `reasoning_content` when both are present. Adds streaming tests for both cases.
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
Sorry, something went wrong.
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
Sorry, something went wrong.
|
The following comment was made by an LLM, it may be inaccurate: Based on my search, I found the following related PRs that are worth reviewing: Related PRs (same domain, different scope):
None of these are direct duplicates of PR #35187, but they are all related to reasoning field handling in OpenAI Chat streaming. PR #34698 appears most closely related as it also modifies reasoning chunk handling in the same protocol. |
Sorry, something went wrong.
|
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
Sorry, something went wrong.
Issue for this PR
No existing issue — standalone bug fix.
Closes #
Type of change
What does this PR do?
The OpenAI Chat streaming delta schema (OpenAIChatDelta in
packages/llm/src/protocols/openai-chat.ts) only declares reasoning_content.
Each SSE chunk is decoded with Protocol.jsonEvent → Schema.fromJsonString,
which strips any key that isn't on the struct. So when a provider streams its
thinking under reasoning instead of reasoning_content, that key is dropped
during decoding and step() never sees it — the reasoning is silently lost.
I ran into this with Scaleway; it also affects self-hosted vLLM / SGLang
deployments and OpenRouter's reasoning field. The model was clearly reasoning
but nothing surfaced in the stream. I had been working around it by rewriting
reasoning → reasoning_content in a proxy sitting in front of
opencode serve; this patch makes that workaround unnecessary.
Two changes:
reasoning_content still takes precedence when a provider sends both
It's additive to the wire schema — providers that only send reasoning_content
(DeepSeek etc.) hit the same code path as before and are unaffected.
How did you verify your code works?
Added two streaming tests in packages/llm/test/provider/openai-chat.test.ts,
alongside the existing reasoning_content one:
reasoning-start / reasoning-delta / reasoning-end events
bun test in packages/llm: 29 pass / 0 fail. bun typecheck is clean (the
pre-push hook also ran the full monorepo typecheck, 30/30).
Screenshots / recordings
N/A — not a UI change.
Checklist