Skip to content

Add next-steps graphs: dynamic context after every appstore call#89

Merged
Alexgodoroja merged 5 commits into
mainfrom
feat/next-steps-graph
Jul 16, 2026
Merged

Add next-steps graphs: dynamic context after every appstore call#89
Alexgodoroja merged 5 commits into
mainfrom
feat/next-steps-graph

Conversation

@Alexgodoroja

@Alexgodoroja Alexgodoroja commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

What & why

Autonomous agents install an app and stop caring, because nothing in a call's output tells them where the flow goes. Product demos fixed that at install time — one banner, once. This fixes it at call time, continuously: every pilotctl appstore call now ends by naming the small set of recommended next commands for where the agent stands — the next step in the flow on success, and the specific fix on failure.

$ pilotctl appstore call io.pilot.sqlite sqlite.query '{"sql":"SELECT 42"}'
error: ipc: server error: backend: missing required param(s): database
hint:  the app "io.pilot.sqlite" rejected or could not handle "sqlite.query"
next: sqlite needs an explicit database — there is no default
  1. pilotctl appstore call io.pilot.sqlite sqlite.query '{"database":":memory:","sql":"SELECT 42 AS a"}'
     why: pass database (:memory: for a scratch db) (fixes the error above)

Authored in submission.json beside product_demo, carried verbatim into the catalogue's metadata.json, cached by pilotctl at install. The client half is pilot-protocol/pilotprotocol#396.

What's here

  • internal/nextsteps — the frozen contract: Graph/Edge/Step, Resolve, Validate, renderers. A flat edge list, not a node tree: the graph is implicit in from → then, which keeps authoring, diffing and testing to one list, and avoids a second thing to keep in sync with exposes.
  • Wiring: next_steps flows submission → config → metadata.json, mirroring product_demo's four touchpoints. Additive and optional — older clients ignore it.
  • CI gates: a graph can never recommend a method the app does not expose; a named gateway must be reachable from a wildcard edge; cross-app steps resolve against real submissions; the field survives ToConfig+BuildMetadata.
  • 19 submissions backfilled — 91 edges, every graph derived from live <ns>.help + real execution.
  • docs/NEXT-STEPS-GRAPHS.md; publish-rich-from-r2.sh carries next_steps into metadata, so a content fix is a metadata republish with no app rebuild.

Two things the ground truth changed

Both were found by running all 19 apps, and both invalidate the obvious design.

1. match is over the outcome PAYLOAD, not the outcome. The most important case is not an error: the scaffold's requireKey soft-fails an unauthenticated call with exit 0 and {"needs_signup":true}. The CLI adapter likewise wraps a non-zero tool exit into a successful call with {"exit":1,"stderr":...} — server-not-started for postgres/mysql/redis is exit 0, an aegis block is exit 0, ideon-free's unknown jobId is exit 0. An outcome-only matcher would have been silently inert on nearly every app. So match tests the error text on failure and the result body on success.

2. A discriminated edge beats a bare exact-from edge. Scoring originally ranked from-exact above a match, so for any wrapped method that also had a flow edge, the flow edge shadowed the gateway: a cold agent calling primitive.send_email was told to read its inbox instead of to sign up — the exact failure this feature exists to prevent, and invisible to CI because both edges are individually valid. The rule is now: an edge that matched the actual situation beats one that merely matched the method name. Regression-tested.

Recommended, not exhaustive

A graph is not <ns>.help. io.pilot.primitive exposes 111 methods; its graph names four. Utility methods that advance the flow only — no getters, no listers. ≤3 steps per edge.

Deliberately not here

Verification

  • go test ./internal/... — 11/11 packages green; gofmt/go vet clean.
  • Every authored command was run for real; graphs were rendered end-to-end against the live daemon.
  • Metered apps were authored without spending: 402/429 semantics come from the broker source, matched against verbatim error strings.

Known limitation

pilotctl caches the graph at install and refreshes on upgrade (which re-runs install --force), but outdated compares app versions — a graph-only fix doesn't bump one, so existing installs won't see it until the app's next version bump. Explicit refresh today is pilotctl appstore install <id> --force. Documented; the fix is to teach outdated to compare metadata_sha256, deliberately left out of this cut.

Alexgodoroja added 4 commits July 15, 2026 14:19
Agents install apps and stop caring, because nothing in a call's output says
where the flow goes. The product demo covers install->first-call, once. A
next-steps graph covers every call after it: each call ends by naming the small
set of recommended next commands for where the agent now stands.

Authored in submission.json beside product_demo, carried verbatim into the
catalogue's metadata.json, cached by pilotctl at install. Recommended, not
exhaustive: primitive exposes 111 methods and its graph names four.

- internal/nextsteps: the frozen contract (Graph/Edge/Step), Resolve, Validate,
  and renderers. A flat edge list, not a node tree: the graph is implicit in
  from->then, which keeps authoring, diffing and testing to one list.
- publish/scaffold wiring, mirroring product_demo's four touchpoints.
- CI gates: a graph can never recommend a method the app does not expose; a
  named gateway must be reachable from a wildcard edge; cross-app steps resolve.
- 19 submissions backfilled (91 edges), every graph derived from live output
  and every authored command actually run.
- docs/NEXT-STEPS-GRAPHS.md; publish-rich-from-r2.sh carries next_steps into
  metadata, so a content fix ships as a metadata republish with no app rebuild.

Two things the ground truth changed, both found by running the real system:

Match is over the outcome PAYLOAD, not the outcome. The most important case is
not an error: the scaffold's requireKey soft-fails an unauthenticated call with
exit 0 and {"needs_signup":true}, and the CLI adapter likewise wraps a non-zero
tool exit into a successful call with {"exit":1,"stderr":...}. Server-not-started
for postgres/mysql/redis is exit 0. An outcome-only matcher would have been
silently inert on nearly every app.

A discriminated edge beats a bare exact-from edge. Scoring originally ranked
from-exact above a match, so for any wrapped method that also had a flow edge,
the flow edge shadowed the gateway: a cold agent calling primitive.send_email
was told to read its inbox instead of to sign up. Regression-tested.

Also fixes io.pilot.aegis, whose cli.args repeated the binary name as argv[0]
("aegis aegis scan") and left 6 of 7 methods returning a usage banner.
The graph is authored in submission.json but consumed from the catalogue's
metadata.json — different structs, different packages, joined by ToConfig +
BuildMetadata. A field that validates and then silently fails to travel would
be invisible to every other gate and would ship as 'the feature does nothing'.
It was labelled 'recovery' only to satisfy the old gateway-reachability gate,
which accepted a wildcard route to a gateway only on an err edge. Miren never
errors — a no-cluster host returns exit 0 with the message in the payload — so
the honest kind was unreachable. The gate now accepts a wildcard edge of either
outcome, so the label can match reality.
The argv fix (dropping the duplicated binary name) ships on its own branch off
main, so it is not held hostage to this feature and this PR stays purely
next-steps. The aegis graph here deliberately routes via aegis.exec, which is
correct for the app as it is published TODAY.
@Alexgodoroja
Alexgodoroja force-pushed the feat/next-steps-graph branch from 4c6b186 to 8a32846 Compare July 15, 2026 21:38
@Alexgodoroja
Alexgodoroja merged commit b730a3c into main Jul 16, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant