Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,12 @@ make validate-examples # validate all examples
- **executor/**: Agent execution
- `agent.py` - `AgentExecutor` handles prompt rendering, tool resolution, and output validation for single agents
- `script.py` - `ScriptExecutor` runs shell commands as workflow steps, capturing stdout/stderr/exit_code
- `wait.py` - `WaitExecutor` pauses workflow execution for a parsed duration via `asyncio.sleep`. Races the sleep against the engine's `interrupt_event` so Esc/Ctrl+G cancels in-flight waits immediately; the workflow-level `limits.timeout_seconds` also cancels it via `LimitEnforcer.wait_for_with_timeout`. Output contract is strictly `{"waited_seconds": float}` per issue #218.
- `template.py` - Jinja2 template rendering
- `output.py` - JSON output parsing and schema validation

- **duration.py**: `parse_duration(value)` shared helper. Accepts plain `int`/`float` seconds, or strings with `ms`/`s`/`m`/`h` suffix. Raises `ValueError` (nests cleanly inside Pydantic `ValidationError`). Rejects booleans. Bounds enforcement (e.g. > 0, 24h cap) lives in callers so the parser can be reused.

- **providers/**: SDK provider abstraction
- `base.py` - `AgentProvider` ABC defining `execute()`, `validate_connection()`, `close()`
- `copilot.py` - GitHub Copilot SDK implementation
Expand All @@ -113,12 +116,13 @@ make validate-examples # validate all examples

1. CLI parses YAML via `config/loader.py` → `WorkflowConfig`
2. `WorkflowEngine` initializes with config and provider
3. Engine loops: find agent/parallel/for-each/script → execute → evaluate routes → next
3. Engine loops: find agent/parallel/for-each/script/wait → execute → evaluate routes → next
4. Parallel groups execute agents concurrently with context isolation (deep copy snapshot)
5. For-each groups resolve source arrays at runtime, inject loop variables (`{{ item }}`, `{{ _index }}`, `{{ _key }}`)
6. Script steps run shell commands via asyncio subprocess, expose stdout/stderr/exit_code to context
7. Routes evaluated via `Router` using Jinja2 or simpleeval expressions
8. Final output built from templates in `output:` section
7. Wait steps pause via `asyncio.sleep` (cancellable by interrupt or workflow timeout); expose `{"waited_seconds": float}` to context
8. Routes evaluated via `Router` using Jinja2 or simpleeval expressions
9. Final output built from templates in `output:` section

### Key Patterns

Expand Down
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased](https://github.com/microsoft/conductor/compare/v0.1.17...HEAD)

### Added
- New `type: wait` workflow step that pauses execution for a parsed
duration via in-process `asyncio.sleep`. Cross-platform — no shell
`sleep` dependency. Use for rate-limit cooldowns, polling intervals,
external-system catch-up, and demos. The `duration:` field accepts
plain numbers (seconds), suffixed strings (`"500ms"`, `"60s"`,
`"2.5m"`, `"1h"`), or a Jinja2 template that renders to one of
those (e.g. `"{{ workflow.input.poll_interval }}s"`). Schema enforces
`0 < duration <= 24h` and rejects boolean values pre-coercion.
`Esc` / `Ctrl+G` cancels in-progress waits immediately (the engine
races the sleep against the interrupt event), and the workflow-level
`limits.timeout_seconds` also cancels them. Wait steps emit
`wait_started` / `wait_completed` / `wait_failed` events alongside
the generic `agent_started` (with `agent_type: "wait"`), so existing
dashboards keyed on agent lifecycle pick them up automatically. The
dashboard adds a dedicated `WaitNode` (clock icon) and `WaitDetail`
panel that show the requested duration, actual elapsed time, reason,
and an "interrupted" indicator. The public output contract is strict
— only `{"waited_seconds": float}` is exposed to workflow context;
extra metadata lives in event payloads. Wait steps count toward
`limits.max_iterations` (each pause is one step) but are not subject
to `max_agent_iterations` (per-LLM-agent tool counter). Wait cannot
be used inside `parallel` or `for_each` groups. New `examples/wait-step.yaml`
demonstrates a polling pattern with a templated poll interval and
route loop-back
([#224](https://github.com/microsoft/conductor/pull/224),
closes [#218](https://github.com/microsoft/conductor/issues/218)).

## [0.1.17](https://github.com/microsoft/conductor/compare/v0.1.16...v0.1.17) - 2026-05-21

### Added
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,7 @@ See the [`examples/`](./examples/) directory for complete workflows:
| [parallel-research.yaml](./examples/parallel-research.yaml) | Static parallel execution |
| [design-review.yaml](./examples/design-review.yaml) | Human gate with loop pattern |
| [script-step.yaml](./examples/script-step.yaml) | Script step with exit_code routing |
| [wait-step.yaml](./examples/wait-step.yaml) | Wait step + script for a polling loop-back pattern |

**More examples and running instructions:** [examples/README.md](./examples/README.md)

Expand Down
4 changes: 2 additions & 2 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,8 +159,8 @@ agents:

Per-agent overrides always win over the workflow-wide default. The
`reasoning.effort` field is **only** valid on standard `agent`-type agents; it
is rejected on `script`, `human_gate`, and `workflow` agents (which do not call
a model).
is rejected on `script`, `human_gate`, `workflow`, and `wait` agents (which do
not call a model).

### Per-provider translation

Expand Down
70 changes: 68 additions & 2 deletions docs/workflow-syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ Agents are defined in the `agents` list. Each agent represents a unit of work.
agents:
- name: string # Required: Unique agent identifier
description: string # Optional: Purpose description
type: agent # agent | human_gate | script | workflow (default: agent)
type: agent # agent | human_gate | script | workflow | wait (default: agent)
model: string # Optional: Model identifier (e.g., 'claude-sonnet-4.5')

prompt: | # Required for type=agent: Agent instructions
Expand Down Expand Up @@ -281,6 +281,71 @@ routes:

**Environment variable note** — values in `env` are passed as-is to the subprocess (they are not rendered as Jinja2 templates). Use `${VAR}` syntax in the workflow YAML loader if you need environment variable substitution in env values.

### Wait Steps

Wait steps pause workflow execution for a parsed duration via in-process `asyncio.sleep`. Use them for rate-limit cooldowns, polling intervals, and external-system catch-up — cross-platform, no shell `sleep` dependency.

```yaml
agents:
- name: cooldown
type: wait
description: "Cool down between API bursts" # Optional
duration: 60s # Required: see "Duration format" below
reason: "Avoiding rate limit" # Optional: shown in dashboard
routes:
- to: next_step
```

**Duration format** — `duration` accepts:

- A plain `int` or `float` (seconds): `duration: 60`, `duration: 1.5`.
- A string with a unit suffix: `ms` (milliseconds), `s` (seconds), `m` (minutes), `h` (hours). Examples: `"500ms"`, `"60s"`, `"2.5m"`, `"1h"`.
- A Jinja2 template that renders to one of the above. Templated durations defer literal validation to runtime:

```yaml
duration: "{{ workflow.input.poll_interval_seconds }}s"
```

The resolved duration must be **greater than 0 and no more than 24 hours** (`86400s`). Longer pauses should reconsider `workflow.limits.timeout_seconds` first.

**Output structure** — wait step output is strict — only `waited_seconds` is exposed:

| Field | Type | Description |
|-------|------|-------------|
| `waited_seconds` | `number` | Wall-clock seconds actually slept (may be less than requested on interrupt) |

Access in templates: `{{ cooldown.output.waited_seconds }}`.

**Polling pattern** — wait composes with routing loop-backs to build polling workflows without writing any Python:

```yaml
agents:
- name: check_status
type: script
command: ./poll-status.sh
routes:
- to: process_result
when: "status == 'ready'"
- to: wait_then_retry

- name: wait_then_retry
type: wait
duration: "{{ workflow.input.poll_interval_seconds }}s"
routes:
- to: check_status # loop back

- name: process_result
# ...
```

**Cancellation** — `Esc` / `Ctrl+G` cancels an in-progress wait immediately (the engine races the sleep against the interrupt event). The workflow-level `limits.timeout_seconds` also cancels in-flight waits via the standard timeout path.

**Iteration counting** — wait steps count toward `workflow.limits.max_iterations` (each pause is one step). They are not subject to `max_agent_iterations`, which counts per-LLM-agent tool iterations.

**Restrictions** — wait steps cannot have `prompt`, `model`, `provider`, `tools`, `system_prompt`, `options`, `command`, `args`, `env`, `working_dir`, `timeout`, `workflow`, `input_mapping`, `max_depth`, `max_session_seconds`, `max_agent_iterations`, `retry`, `dialog`, `reasoning`, `timeout_seconds`, or `output`. Wait steps also cannot be used inside `parallel` groups or `for_each` groups.

See [`examples/wait-step.yaml`](../examples/wait-step.yaml) for a complete polling workflow.

### Sub-Workflow Steps

Sub-workflow steps reference external workflow YAML files, enabling composable and reusable workflow building blocks. The sub-workflow runs as a black box — its internal agents are not visible to the parent.
Expand Down Expand Up @@ -401,7 +466,7 @@ After the conversation, the agent re-executes with the dialog transcript as addi
| `dialog.trigger_prompt` | string | Yes | Criteria for the LLM evaluator to decide when dialog is needed |

**Behavior notes:**
- Dialog is supported on regular `agent` type only (not `human_gate`, `script`, or `workflow`)
- Dialog is supported on regular `agent` type only (not `human_gate`, `script`, `workflow`, or `wait`)
- In web dashboard mode, the dialog temporarily replaces the graph area with a chat interface
- When `--skip-gates` is set (e.g., CI/automation), dialogs are automatically skipped
- The evaluator prompt should describe *when* to trigger dialog, not *what* to ask — the evaluator generates the opening question from the agent's output context
Expand Down Expand Up @@ -695,6 +760,7 @@ workflow:
- Each agent execution counts as 1 iteration
- Parallel agents count individually (3 parallel agents = 3 iterations)
- Loop-back patterns increment the counter on each iteration
- Script steps and wait steps each count as 1 iteration

### Timeout Behavior

Expand Down
28 changes: 28 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,34 @@ Research workflow demonstrating multi-provider patterns. Demonstrates:
conductor run examples/multi-provider-research.yaml --input topic="Cloud computing"
```

## Step Types

### script-step.yaml

Script step with shell command, JSON output parsing, and `exit_code`-based routing.
Demonstrates:
- `type: script` agents (cross-platform shell command execution)
- Capturing stdout/stderr/exit_code
- Routing on `exit_code` (`when: "exit_code == 0"`)
- Passing script output to downstream LLM agents

```bash
conductor run examples/script-step.yaml
```

### wait-step.yaml

Polling pattern with a wait step and a routing loop-back. Demonstrates:
- `type: wait` agents (pure `asyncio.sleep`, cross-platform — no shell `sleep` dependency)
- Templated `duration` (`"{{ workflow.input.poll_interval_seconds }}s"`)
- Loop-back from wait → script for polling
- `reason` field surfaced in the dashboard

```bash
conductor run examples/wait-step.yaml \
--input poll_interval_seconds=2 --input max_attempts=3
```

## Planning and Implementation

### plan.yaml
Expand Down
104 changes: 104 additions & 0 deletions examples/wait-step.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Wait Step Example
#
# This example demonstrates the `type: wait` step. A wait step pauses
# workflow execution for a parsed duration (seconds, "Ns/Nm/Nh/Nms",
# or a Jinja2 template that evaluates to one of those). The pause is
# pure in-process asyncio.sleep — no shell `sleep` command, so it works
# cross-platform.
#
# This workflow demonstrates a common pattern: polling an external
# system until it returns a "ready" status, with a cool-down between
# attempts. The wait step's duration is templated from a workflow
# input so the caller controls the poll cadence.
#
# Usage:
# conductor run examples/wait-step.yaml \
# --input poll_interval_seconds=2 --input max_attempts=3

workflow:
name: wait-step-demo
description: Polling pattern with a wait step and a routing loop-back
version: "1.0.0"
entry_point: check_status

runtime:
provider: copilot

input:
poll_interval_seconds:
type: number
default: 5
description: Seconds to wait between polling attempts.
max_attempts:
type: number
default: 3
description: Maximum number of polling attempts before giving up.

limits:
# Wait steps count toward max_iterations (each pause is one step),
# so allow enough headroom for max_attempts polls + waits + summary.
max_iterations: 20
timeout_seconds: 600

agents:
# A trivial script that simulates polling an external system. In a
# real workflow this would hit an HTTP endpoint, query a job queue,
# check a CDN, etc. Here we just bump a counter and report "ready"
# on the third attempt.
- name: check_status
type: script
description: Poll the (simulated) external system for readiness.
command: python3
args:
- "-c"
- |
import json, os, pathlib
# Use a small counter file in /tmp so reruns are independent.
p = pathlib.Path(os.environ.get("STATUS_FILE", "/tmp/wait-step-demo.count"))
count = int(p.read_text()) if p.exists() else 0
count += 1
p.write_text(str(count))
ready = count >= int(os.environ.get("READY_AFTER", "3"))
print(json.dumps({
"status": "ready" if ready else "pending",
"attempt": count,
}))
env:
STATUS_FILE: /tmp/wait-step-demo.count
READY_AFTER: "{{ workflow.input.max_attempts }}"
routes:
- to: summarize
when: "status == 'ready'"
- to: cool_down

# Wait for the configured poll interval, then loop back to check_status.
# The duration template uses workflow.input.* directly — wait steps
# render locally (no LLM), so workflow.input is available without an
# explicit `input:` declaration.
- name: cool_down
type: wait
duration: "{{ workflow.input.poll_interval_seconds }}s"
reason: "Cooling down between polling attempts"
routes:
- to: check_status

# Summarize the polling result. An ordinary agent reading the most
# recent check_status output and reporting back to the caller.
- name: summarize
description: Summarize the polling result.
model: claude-haiku-4.5
input:
- "check_status.output"
prompt: |
The poll loop reported readiness on attempt {{ check_status.output.attempt }}.
Write a single short sentence summarizing the result.
output:
summary:
type: string
description: One-sentence summary of the polling result.
routes:
- to: $end

output:
result: "{{ summarize.output.summary }}"
attempts: "{{ check_status.output.attempt }}"
1 change: 1 addition & 0 deletions plugins/conductor/skills/conductor/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ For runtime config, context modes, limits, and cost tracking, see [references/au
| `entry_point` | First agent/group to execute |
| `routes` | Where agent goes next (`$end` to finish, `self` to loop) |
| `type: script` | Shell command step (captures stdout, stderr, exit_code; JSON stdout is auto-merged) |
| `type: wait` | Pause via `asyncio.sleep` (cross-platform); duration accepts `Ns/Nm/Nh/Nms` or Jinja2; composes with route loop-backs for polling |
| `type: workflow` | Sub-workflow agent — runs another YAML file as a black box (supports `input_mapping`, `max_depth`) |
| `parallel` | Static parallel groups (fixed agent list) |
| `for_each` | Dynamic parallel groups (runtime-determined array; supports `type: workflow` agents) |
Expand Down
Loading
Loading