Mirror reasoning summaries when requested#43
Conversation
Populate reasoning summary text for opt-in requests so downstream OpenAI-compatible clients can surface reasoning without client-side adapter patches.
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Emit OpenAI-compatible reasoning summary stream events for opt-in requests so PydanticAI can populate ThinkingPart content during streaming.
There was a problem hiding this comment.
Pull request overview
Adds opt-in support for exposing reasoning summaries in an OpenAI Responses-compatible shape, primarily by mirroring raw reasoning into output[].summary[] and emitting corresponding streaming events so downstream clients can consume “thinking” summaries without custom adapters.
Changes:
- Extends request schema to accept additional
reasoning.summarymodes (raw,none). - Streams new reasoning-summary events and mirrors reasoning text into
output[].summary[]when requested. - Closes reasoning-summary stream lifecycle with corresponding
*.doneevents and adds/updates tests.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/schemas.ts | Extends reasoning.summary enum with raw/none (but still includes concise/detailed). |
| src/schemas.test.ts | Updates schema tests to cover summary: "raw" and opt-out via "none". |
| src/routes/responses/innerStream.ts | Validates supported summary modes and forwards the mode into handleOneTurnStream. |
| src/routes/responses/innerStream.test.ts | Updates tests for new validation error text and verifies mode forwarding. |
| src/routes/responses/handleOneTurn.ts | Mirrors reasoning text into summary[] and emits response.reasoning_summary_* streaming events when requested. |
| src/routes/responses/handleOneTurn.test.ts | Adds coverage for mirroring behavior and default empty summaries. |
| src/routes/responses/closeOutputItem.ts | Emits reasoning_summary_* .done events when closing a reasoning item with summaries. |
| src/routes/responses/closeOutputItem.test.ts | Adds test ensuring summary done events are emitted. |
| src/openai_patch.ts | Adds patched event/type definitions for reasoning summary stream events. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
phvalguima
left a comment
There was a problem hiding this comment.
I have the impression the sequence_number will not be followed correctly with this change, can we confirm this is or not the case?
| output_index: responseObject.output.length - 1, | ||
| summary_index: currentReasoningItem.summary.length - 1, | ||
| delta: reasoningText as string, | ||
| sequence_number: SEQUENCE_NUMBER_PLACEHOLDER, |
There was a problem hiding this comment.
Why is response.reasoning_summary_text.done? It feels this if (mirrorRawReasoningToSummary) { should also be done on the closeOutputItem.ts.
Refactor reasoning summary handling to support detailed summaries and adjust related tests. The reasoning summary schema now excludes "raw" and "none" options, and tests have been updated to reflect these changes, ensuring correct behavior for omitted summaries and detailed reasoning. Additionally, new tests verify the default behavior of reasoning summary events.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
phvalguima
left a comment
There was a problem hiding this comment.
I think these issues still apply: #43 (comment)
| output_index: responseObject.output.length - 1, | ||
| summary_index: summaryIndex, | ||
| part, | ||
| sequence_number: SEQUENCE_NUMBER_PLACEHOLDER, |
There was a problem hiding this comment.
What happened with this check?
The SEQUENCE_NUMBER_PLACEHOLDER is not getting updated. So, this problem still applies:
response.reasoning_text.done
REASONING_TEXT
sequence_number: 10
response.reasoning_summary_text.done
REASONING_TEXT
sequence_number: 10
There was a problem hiding this comment.
I did look into it, and it seems to work fine and ran the tests fine as well.
so from my limited understanding of the code base and with the help of cursor, the inner stream uses SEQUENCE_NUMBER_PLACEHOLDER only as a local sentinel; runCreateResponseStream rewrites sequence_number for every emitted event with a monotonic counter. I added a regression test (rewrites inner placeholder sequence numbers so reasoning events never collide) that simulates two inner events both set to -1 and verifies the streamed events have unique, strictly increasing sequence numbers, so those reasoning events can’t collide on the wire.
Summary
reasoning.summary.reasoning.summary: "auto"or"raw", raw reasoning text is mirrored intooutput[].summary[]assummary_textwhile preserving the existing rawcontent[]reasoning payload.response.reasoning_summary_part.*andresponse.reasoning_summary_text.*) so streaming clients can consume summaries as they arrive.Why
PydanticAI intentionally treats raw chain-of-thought differently from reasoning summaries. Raw reasoning in
content[]is stored inprovider_details["raw_content"], so the default AG-UI adapter does not show it and PydanticAI's OTEL serializer does not include it in Langfuse traces.By emitting summaries in the same shape OpenAI Responses uses, PydanticAI can populate
ThinkingPart.contentwithout app-specific workarounds. This should let clients like tiger-chat remove the customRawCoTAdapterpath and avoid separate Langfuse/OTEL hacks once they opt in withopenai_reasoning_summary="auto".Compatibility
reasoning.summary,summary[]remains empty and existing raw reasoning events continue to stream.reasoning.summary, so existing callers should not see reasoning exposed differently unless they request it."none"is accepted as an explicit opt-out; unsupported summary modes still fail fast.Test plan
pnpm --dir /Users/mohamed.ali/repos/aleph-alpha/responses.js run test -- src/routes/responses/handleOneTurn.test.ts src/routes/responses/closeOutputItem.test.ts src/routes/responses/innerStream.test.ts src/schemas.test.tspnpm --dir /Users/mohamed.ali/repos/aleph-alpha/responses.js run checkpnpm --dir /Users/mohamed.ali/repos/aleph-alpha/responses.js exec prettier --check src/openai_patch.ts src/schemas.ts src/schemas.test.ts src/routes/responses/innerStream.ts src/routes/responses/innerStream.test.ts src/routes/responses/handleOneTurn.ts src/routes/responses/handleOneTurn.test.ts src/routes/responses/closeOutputItem.ts src/routes/responses/closeOutputItem.test.tsref: https://aleph-alpha.atlassian.net/wiki/spaces/CHAT/pages/2507604161/CHAT-RFC-011+Reasoning+visibility+with+Stateful+Responses+API