Wolo/output validation#970
Draft
wolo-lab wants to merge 2 commits into
Draft
Conversation
ba4809e to
dce4f28
Compare
60a2bf4 to
e02e84a
Compare
43c93bd to
656ebe4
Compare
Completes the symmetric input/output validation contract on the Node interface, alongside the existing ValidateInput. The scheduler is expected to invoke ValidateOutput on every yielded event whose output is non-nil before forwarding the event to the consumer (wired up in a follow-up). Interface and conformance - Add ValidateOutput(output any) (any, error) to the Node interface. - Add explicit stubs on the two implementations that do not embed BaseNode: startNode and the test-only dummyNode. - Extend the compile-time Node-conformance assertions in base_node_test.go to cover AgentNode, ToolNode, JoinNode, ParallelWorker, and WorkflowNode. Default implementation on BaseNode - BaseNode.ValidateOutput delegates to a shared defaultValidateOutput helper that validates the output against the node's outputSchema field (added in #911) when set, otherwise returns the output unchanged. - The default deliberately performs no type coercion or Content/JSON fallback handling; ToolNode will override ValidateOutput to add its FunctionTool {"result": X} unwrap fallback in a follow-up. Tests - Extend TestBaseNode_NilSchemas and TestBaseNode_WithSchemas to cover ValidateOutput symmetrically with ValidateInput: nil-schema passthrough, with-schema delegation (valid output passes, invalid output fails). BUG=516381573 BUG=516383075 (cherry picked from commit 9676bff)
…llback Implements the runtime side of the workflow output-validation pipeline: the scheduler now invokes node.ValidateOutput on every yielded event whose Output is non-nil, before the value is committed to the node-run accumulator and forwarded to the consumer. A validation failure is recorded on the accumulator and surfaces as NodeFailed via the normal completion path. Helper (defaultValidateOutput) - Extracted from base_node.go into a new workflow/output_validation.go to make room for the fallback strategy. - Adds a passthrough short-circuit for framework control values (*session.Event, *session.RequestInput) that are routed through Event.Output by some nodes but are not user output payloads and must never be schema-validated. - Adds a *genai.Content fallback: when standard validation fails on a *genai.Content value, extract concatenated text from the parts; if the schema's root type is "string" return the text directly, else JSON-parse the text and re-validate the parsed value. Mirrors the Python ADK _validate_output_data behavior and gives LlmAgent-like nodes that yield raw model output a deterministic projection onto their declared output schema. - On total failure, the original validation error is returned so callers see the schema mismatch and not a downstream JSON-parse error. Scheduler - handleEvent now calls a new s.validateNodeOutput helper for events with ev.Output != nil. On success the (possibly coerced) value is written back to ev.Output and accumulated; on failure the wrapped error is recorded via nr.recordErr and the activation transitions to NodeFailed. - Events without Output (progress/status events, RequestedInput-only events, routing-only events) bypass validation entirely. Tests - New workflow/output_validation_test.go covers the helper end-to-end: nil schema passthrough, valid payload, invalid payload, both passthrough types, Content fallback for string schemas, Content fallback with JSON parse, invalid JSON falling back to the original error, and empty-text falling back to the original error. - New scheduler tests cover the three required cases: yielded output that validates is forwarded to the consumer; output that fails validation surfaces a wrapped "output validation failed for node X" error and ends the activation; progress events without Output are forwarded without ValidateOutput being invoked. BUG=516382302 BUG=516381927 (cherry picked from commit 69e2510)
e02e84a to
d8a39cd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
2. Or, if no issue exists, describe the change:
If applicable, please follow the issue templates to provide as much detail as
possible.
Problem:
A clear and concise description of what the problem is.
Solution:
A clear and concise description of what you want to happen and why you choose
this solution.
Testing Plan
Please describe the tests that you ran to verify your changes. This is required
for all PRs that are not small documentation or typo fixes.
Unit Tests:
Please include a summary of passed go test results.
Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any
necessary setup or configuration. Please provide logs or screenshots to help
reviewers better understand the fix.
Checklist
Additional context
Add any other context or screenshots about the feature request here.