Skip to content

[Platform] Add streaming support for structured output#2116

Draft
wachterjohannes wants to merge 6 commits into
symfony:mainfrom
wachterjohannes:feature/streaming-structured-output
Draft

[Platform] Add streaming support for structured output#2116
wachterjohannes wants to merge 6 commits into
symfony:mainfrom
wachterjohannes:feature/streaming-structured-output

Conversation

@wachterjohannes

Copy link
Copy Markdown
Contributor
Q A
Bug fix? no
New feature? yes
Docs? yes
Issues -
License MIT

Note

Stacked on top of #2113 (PartialJsonParser). The diff includes the parser commit until #2113 merges into main.

This PR wires PartialJsonParser into the platform so callers can use stream: true together with response_format: SomeClass::class and consume progressively populated typed objects as the model emits more tokens.

Usage

$result = $platform->invoke('gpt-4o-mini', $messages, [
    'stream' => true,
    'response_format' => City::class,
]);

foreach ($result->asObjectStream() as $delta) {
    render($delta->getObject()); // partially populated City instance
}

$final = $result->asObject(); // fully materialized, validated City

What's included

  • PartialObjectDelta (src/platform/src/Result/Stream/Delta/PartialObjectDelta.php) — new DeltaInterface carrying the typed partial object + the raw JSON buffer accumulated so far.
  • PartialObjectStreamListener (src/platform/src/StructuredOutput/Streaming/PartialObjectStreamListener.php) — extends AbstractStreamListener. Buffers TextDelta payloads, runs them through PartialJsonParser, hashes the parsed array (md5(json_encode(...))) to dedup, denormalizes via the same StructuredOutput\Serializer used by the non-streaming path, and rewrites the DeltaEvent to a \Generator that yields both the original TextDelta and a new PartialObjectDelta. On onComplete() it captures the final ObjectResult for later retrieval. If a ValidatorInterface is injected, it validates only the final object — partials are never validated.
  • StructuredOutput\ResultConverter — new branch for StreamResult + outputType: attaches the listener to the stream and returns it unchanged. Constructor signature for $serializer widened to SerializerInterface&DenormalizerInterface (the default StructuredOutput\Serializer already implements both).
  • StructuredOutput\Validator\ValidatorResultConverter — new branch: when inner returns a StreamResult with a PartialObjectStreamListener attached, inject the validator so completion-time validation runs as expected.
  • PlatformSubscriber — the stream: true InvalidArgumentException is dropped; constructor's $serializer widened to the same intersection.
  • StreamResult — public getListeners(): ListenerInterface[] accessor so DeferredResult and the validator converter can discover the partial-object listener.
  • DeferredResult — new asObjectStream(): \Generator<PartialObjectDelta> accessor mirroring asTextStream(). asObject() is extended: if the converted result is a StreamResult with a PartialObjectStreamListener attached, it drains the stream once to fire onComplete() and returns the listener's cached final object (idempotent).
  • 34 tests covering the new units: PartialObjectDeltaTest, PartialObjectStreamListenerTest (yields per snapshot, still yields original TextDeltas, dedup, captures final, validator runs only on final, validator rejects invalid final), extended DeferredResultTest (asObjectStream, asObject after stream, idempotency, short-circuit after asObjectStream), extended ResultConverterTest, ValidatorSubscriberTest (partials not validated, final validated), PlatformSubscriberTest (streaming + structured output is now allowed).

Constraints & decisions

  1. Delta wrapper, not raw objects — keeps shape parity with TextDelta and lets us also carry the raw buffer for debugging.
  2. Listener attached inside StructuredOutput\ResultConverter::convert() — mirrors how the non-streaming path wraps results, follows the MetaDataStreamListener precedent.
  3. md5(json_encode(...)) dedup before denormalization — cheap, deterministic; avoids running the serializer when the parsed shape hasn't changed.
  4. Validation only on the final object — partial snapshots are by definition incomplete, so running constraints over them would produce false positives.
  5. Emit both TextDelta and PartialObjectDelta — the listener replaces the current delta with a \Generator (already supported by DeltaEvent::setDelta() and StreamResult::getContent()'s yield from $delta path), so asTextStream() keeps working unchanged.

Validation

  • cd src/platform && vendor/bin/phpunit — 610 tests / 1267 assertions, all green.
  • cd src/platform && vendor/bin/phpstan analyse — clean for the new files (one pre-existing Schema.php:75 unrelated error remains on main).
  • vendor/bin/php-cs-fixer fix --dry-run — clean.
  • End-to-end manual: OPENAI_API_KEY=… php examples/platform/streaming-structured-output.php (see new example).

Adds a static, dependency-free utility that recovers the largest valid
structure from an incomplete JSON string. This unblocks rendering
partial objects from streamed structured output before the model
finishes emitting them.

The implementation handles trailing commas, unclosed strings, dangling
colons, partial true/false/null literals, and unclosed object/array
structures. Ported from modelflow-ai's OptimisticJsonParser.

Includes example, documentation under Structured Output, and an entry
in the platform CHANGELOG.
Lifts the previous restriction that prevented combining stream: true
with response_format: SomeClass::class. The new PartialObjectStreamListener
buffers TextDelta payloads, runs them through PartialJsonParser, and
denormalizes the recovered structure into the target class using the
same StructuredOutput Serializer as the non-streaming path.

Consumers iterate progressively populated typed objects via
DeferredResult::asObjectStream(), which yields PartialObjectDelta
instances (carrying the object + raw buffer). The original TextDelta
stream remains intact for callers that still want raw text via
asTextStream(). DeferredResult::asObject() drains the stream once and
returns the final, validated object — partial snapshots are never
validated, since they are by definition incomplete.

Includes example, documentation, and CHANGELOG entry.
@wachterjohannes wachterjohannes force-pushed the feature/streaming-structured-output branch from a85eb89 to 1895759 Compare May 27, 2026 07:38
@chr-hertel

Copy link
Copy Markdown
Member

holy. oh my. impressive.

i honestly never looked at this since i thought this is not really necessary - so def pretty cool, curious about the use-case nevertheless.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants