Skip to content

Wolo/output validation#970

Draft
wolo-lab wants to merge 2 commits into
wolo/run_node_use_as_output_samplefrom
wolo/output-validation
Draft

Wolo/output validation#970
wolo-lab wants to merge 2 commits into
wolo/run_node_use_as_output_samplefrom
wolo/output-validation

Conversation

@wolo-lab
Copy link
Copy Markdown

@wolo-lab wolo-lab commented Jun 6, 2026

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

  • Closes: #issue_number
  • Related: #issue_number

2. Or, if no issue exists, describe the change:

If applicable, please follow the issue templates to provide as much detail as
possible.

Problem:
A clear and concise description of what the problem is.

Solution:
A clear and concise description of what you want to happen and why you choose
this solution.

Testing Plan

Please describe the tests that you ran to verify your changes. This is required
for all PRs that are not small documentation or typo fixes.

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Please include a summary of passed go test results.

Manual End-to-End (E2E) Tests:

Please provide instructions on how to manually test your changes, including any
necessary setup or configuration. Please provide logs or screenshots to help
reviewers better understand the fix.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

Add any other context or screenshots about the feature request here.

@wolo-lab wolo-lab force-pushed the wolo/run_node_use_as_output_sample branch from ba4809e to dce4f28 Compare June 8, 2026 08:07
@wolo-lab wolo-lab force-pushed the wolo/output-validation branch from 60a2bf4 to e02e84a Compare June 8, 2026 08:08
@wolo-lab wolo-lab force-pushed the wolo/run_node_use_as_output_sample branch 2 times, most recently from 43c93bd to 656ebe4 Compare June 8, 2026 08:18
wolo-lab added 2 commits June 8, 2026 08:19
Completes the symmetric input/output validation contract on the Node
interface, alongside the existing ValidateInput. The scheduler is
expected to invoke ValidateOutput on every yielded event whose output
is non-nil before forwarding the event to the consumer (wired up in a
follow-up).

Interface and conformance
- Add ValidateOutput(output any) (any, error) to the Node interface.
- Add explicit stubs on the two implementations that do not embed
  BaseNode: startNode and the test-only dummyNode.
- Extend the compile-time Node-conformance assertions in
  base_node_test.go to cover AgentNode, ToolNode, JoinNode,
  ParallelWorker, and WorkflowNode.

Default implementation on BaseNode
- BaseNode.ValidateOutput delegates to a shared defaultValidateOutput
  helper that validates the output against the node's outputSchema
  field (added in #911) when set, otherwise returns the output
  unchanged.
- The default deliberately performs no type coercion or Content/JSON
  fallback handling; ToolNode will override ValidateOutput to add its
  FunctionTool {"result": X} unwrap fallback in a follow-up.

Tests
- Extend TestBaseNode_NilSchemas and TestBaseNode_WithSchemas to cover
  ValidateOutput symmetrically with ValidateInput: nil-schema
  passthrough, with-schema delegation (valid output passes, invalid
  output fails).

BUG=516381573
BUG=516383075

(cherry picked from commit 9676bff)
…llback

Implements the runtime side of the workflow output-validation pipeline:
the scheduler now invokes node.ValidateOutput on every yielded event
whose Output is non-nil, before the value is committed to the node-run
accumulator and forwarded to the consumer. A validation failure is
recorded on the accumulator and surfaces as NodeFailed via the normal
completion path.

Helper (defaultValidateOutput)
- Extracted from base_node.go into a new workflow/output_validation.go
  to make room for the fallback strategy.
- Adds a passthrough short-circuit for framework control values
  (*session.Event, *session.RequestInput) that are routed through
  Event.Output by some nodes but are not user output payloads and must
  never be schema-validated.
- Adds a *genai.Content fallback: when standard validation fails on a
  *genai.Content value, extract concatenated text from the parts; if
  the schema's root type is "string" return the text directly, else
  JSON-parse the text and re-validate the parsed value. Mirrors the
  Python ADK _validate_output_data behavior and gives LlmAgent-like
  nodes that yield raw model output a deterministic projection onto
  their declared output schema.
- On total failure, the original validation error is returned so
  callers see the schema mismatch and not a downstream JSON-parse
  error.

Scheduler
- handleEvent now calls a new s.validateNodeOutput helper for events
  with ev.Output != nil. On success the (possibly coerced) value is
  written back to ev.Output and accumulated; on failure the wrapped
  error is recorded via nr.recordErr and the activation transitions
  to NodeFailed.
- Events without Output (progress/status events, RequestedInput-only
  events, routing-only events) bypass validation entirely.

Tests
- New workflow/output_validation_test.go covers the helper end-to-end:
  nil schema passthrough, valid payload, invalid payload, both
  passthrough types, Content fallback for string schemas, Content
  fallback with JSON parse, invalid JSON falling back to the original
  error, and empty-text falling back to the original error.
- New scheduler tests cover the three required cases: yielded output
  that validates is forwarded to the consumer; output that fails
  validation surfaces a wrapped "output validation failed for node X"
  error and ends the activation; progress events without Output are
  forwarded without ValidateOutput being invoked.

BUG=516382302
BUG=516381927

(cherry picked from commit 69e2510)
@wolo-lab wolo-lab force-pushed the wolo/output-validation branch from e02e84a to d8a39cd Compare June 8, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant