[Platform] Add streaming support for structured output#2116
Draft
wachterjohannes wants to merge 6 commits into
Draft
[Platform] Add streaming support for structured output#2116wachterjohannes wants to merge 6 commits into
wachterjohannes wants to merge 6 commits into
Conversation
Adds a static, dependency-free utility that recovers the largest valid structure from an incomplete JSON string. This unblocks rendering partial objects from streamed structured output before the model finishes emitting them. The implementation handles trailing commas, unclosed strings, dangling colons, partial true/false/null literals, and unclosed object/array structures. Ported from modelflow-ai's OptimisticJsonParser. Includes example, documentation under Structured Output, and an entry in the platform CHANGELOG.
Lifts the previous restriction that prevented combining stream: true with response_format: SomeClass::class. The new PartialObjectStreamListener buffers TextDelta payloads, runs them through PartialJsonParser, and denormalizes the recovered structure into the target class using the same StructuredOutput Serializer as the non-streaming path. Consumers iterate progressively populated typed objects via DeferredResult::asObjectStream(), which yields PartialObjectDelta instances (carrying the object + raw buffer). The original TextDelta stream remains intact for callers that still want raw text via asTextStream(). DeferredResult::asObject() drains the stream once and returns the final, validated object — partial snapshots are never validated, since they are by definition incomplete. Includes example, documentation, and CHANGELOG entry.
a85eb89 to
1895759
Compare
Member
|
holy. oh my. impressive. i honestly never looked at this since i thought this is not really necessary - so def pretty cool, curious about the use-case nevertheless. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Stacked on top of #2113 (PartialJsonParser). The diff includes the parser commit until #2113 merges into main.
This PR wires
PartialJsonParserinto the platform so callers can usestream: truetogether withresponse_format: SomeClass::classand consume progressively populated typed objects as the model emits more tokens.Usage
What's included
PartialObjectDelta(src/platform/src/Result/Stream/Delta/PartialObjectDelta.php) — newDeltaInterfacecarrying the typed partial object + the raw JSON buffer accumulated so far.PartialObjectStreamListener(src/platform/src/StructuredOutput/Streaming/PartialObjectStreamListener.php) — extendsAbstractStreamListener. BuffersTextDeltapayloads, runs them throughPartialJsonParser, hashes the parsed array (md5(json_encode(...))) to dedup, denormalizes via the sameStructuredOutput\Serializerused by the non-streaming path, and rewrites theDeltaEventto a\Generatorthat yields both the originalTextDeltaand a newPartialObjectDelta. OnonComplete()it captures the finalObjectResultfor later retrieval. If aValidatorInterfaceis injected, it validates only the final object — partials are never validated.StructuredOutput\ResultConverter— new branch forStreamResult + outputType: attaches the listener to the stream and returns it unchanged. Constructor signature for$serializerwidened toSerializerInterface&DenormalizerInterface(the defaultStructuredOutput\Serializeralready implements both).StructuredOutput\Validator\ValidatorResultConverter— new branch: when inner returns aStreamResultwith aPartialObjectStreamListenerattached, inject the validator so completion-time validation runs as expected.PlatformSubscriber— thestream: trueInvalidArgumentExceptionis dropped; constructor's$serializerwidened to the same intersection.StreamResult— publicgetListeners(): ListenerInterface[]accessor soDeferredResultand the validator converter can discover the partial-object listener.DeferredResult— newasObjectStream(): \Generator<PartialObjectDelta>accessor mirroringasTextStream().asObject()is extended: if the converted result is aStreamResultwith aPartialObjectStreamListenerattached, it drains the stream once to fireonComplete()and returns the listener's cached final object (idempotent).PartialObjectDeltaTest,PartialObjectStreamListenerTest(yields per snapshot, still yields originalTextDeltas, dedup, captures final, validator runs only on final, validator rejects invalid final), extendedDeferredResultTest(asObjectStream,asObjectafter stream, idempotency, short-circuit afterasObjectStream), extendedResultConverterTest,ValidatorSubscriberTest(partials not validated, final validated),PlatformSubscriberTest(streaming + structured output is now allowed).Constraints & decisions
TextDeltaand lets us also carry the raw buffer for debugging.StructuredOutput\ResultConverter::convert()— mirrors how the non-streaming path wraps results, follows theMetaDataStreamListenerprecedent.md5(json_encode(...))dedup before denormalization — cheap, deterministic; avoids running the serializer when the parsed shape hasn't changed.TextDeltaandPartialObjectDelta— the listener replaces the current delta with a\Generator(already supported byDeltaEvent::setDelta()andStreamResult::getContent()'syield from $deltapath), soasTextStream()keeps working unchanged.Validation
cd src/platform && vendor/bin/phpunit— 610 tests / 1267 assertions, all green.cd src/platform && vendor/bin/phpstan analyse— clean for the new files (one pre-existingSchema.php:75unrelated error remains onmain).vendor/bin/php-cs-fixer fix --dry-run— clean.OPENAI_API_KEY=… php examples/platform/streaming-structured-output.php(see new example).