Skip to content

Add use_pipeline step type for pipeline composition#10

Merged
viamus merged 4 commits intomainfrom
feature/pipeline-composition
Mar 1, 2026
Merged

Add use_pipeline step type for pipeline composition#10
viamus merged 4 commits intomainfrom
feature/pipeline-composition

Conversation

@viamus
Copy link
Owner

@viamus viamus commented Mar 1, 2026

Summary

  • New use_pipeline step type that loads and executes another pipeline YAML as a sub-pipeline, enabling pipeline composition and reuse
  • Supports inputs: mapping from parent context (with {{steps.xxx}} template resolution) and output merging back into the parent (with/without output_key, respects child outputs: section)
  • Circular reference detection via AsyncLocal<ImmutableHashSet<string>> — safe across async/parallel boundaries
  • Full optional: true support — child pipeline failure converts to Skipped
  • 26 new tests covering parsing, validation, execution, input/output mapping, error handling, and edge cases

Example usage

steps:
  - name: "Prepare data"
    prompt: "..."
    output_key: data

  - name: "Run analysis"
    use_pipeline: ./pipelines/analysis.yml
    inputs:
      source: "{{steps.data}}"
    output_key: analysis_result

  - name: "Report"
    prompt: "Generate report from {{steps.analysis_result}}"

Test plan

  • dotnet build compiles without errors
  • dotnet test — all 137 tests pass (26 new)
  • Manual: dotnet run --project CodeGenesis.Engine -- run-pipeline examples/use-pipeline/main.yml

🤖 Generated with Claude Code

viamus and others added 4 commits March 1, 2026 11:32
Enable referencing and executing another pipeline YAML as a sub-pipeline,
passing inputs from the parent and receiving outputs back. This allows
pipelines to be composed as reusable building blocks.

- Add UsePipeline and Inputs fields to StepEntry with YAML aliases
- Add validation for use_pipeline steps in PipelineConfigLoader
- Create UsePipelineStep composite step with circular reference detection
  via AsyncLocal<ImmutableHashSet<string>>, isolated child context,
  input mapping, output merging, and optional support
- Wire into StepBuilder and PipelineExecutor dispatch
- Add sub-pipeline rendering methods to PipelineRenderer
- Add 26 tests (StepEntry discriminators, validation, UsePipelineStep)
- Add example use-pipeline YAML files (main.yml + analysis.yml)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Upgrade Spectre.Console to 0.54/0.53.1 and MS Extensions to 10.0.3
- Adapt to Spectre.Console.Cli CancellationToken breaking change
- Treat empty/whitespace model strings as null to prevent blank --model args
- Improve sub-pipeline completion rendering with step count
- Pin .NET SDK 10.0.103 via global.json
- Add publish profile for self-contained win-x64 builds

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Match the exact SDK version from global.json instead of using preview quality channel.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Publish profiles contain local paths and should not be in source control.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@viamus viamus merged commit 6e2d880 into main Mar 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant