feat(workflows): Templates gallery + PR code-review automation#75
feat(workflows): Templates gallery + PR code-review automation#75xBalbinus wants to merge 27 commits into
Conversation
Adds a Zapier-style template gallery on top of the dag/v1 workflow interpreter: a worker-side template registry, an install API, and a client gallery. The flagship template reviews a pull request — fetches the diff, reviews the changed lines with an LLM, then posts a review comment back on the PR. - shared: WorkflowTemplateSummary / InstallTemplateResponse types. - worker: template registry + install service (createWorkflow -> saveDraft -> publishDraft -> webhook trigger). GET /api/templates and POST /api/templates/:id/install. Install is atomic — a failure after the workflow row is created rolls it back, and the webhook path/name are suffixed with a fresh uuid so repeat installs don't collide. - plugin-github: opt-in includePatch on github.inspect_pull_request so a reviewer can see the actual diff. Backward-compatible — patch is only present when includePatch is true. - client: /automation/templates gallery with "Use this template", a one-time webhook secret reveal, and a "Run now" dialog (manual run). - test: asserts every template definition passes the publish validator.
- app-logo chain per card (trigger -> ... -> action), reusing the app's
brand logos via getServiceIcon; the LLM step gets a Claude mark
- verb-first, plain-language titles ("Review pull requests and post a
comment") plus short descriptions
- search box + category filter pills over the gallery
- "Try it" opens a guided setup dialog: numbered steps, then
"Use this template" -> run-now inputs + one-time webhook reveal
- template metadata gains apps[] + steps[] to drive the card + setup UI
ee329bd to
48e2046
Compare
|
Preview deployment: https://pr-75.dev-valet-turnkey-client.pages.dev |
Per Conner's suggestion: templates now live in the plugin that owns their actions, and the integration registry aggregates them — exactly like actions/triggers. Enabling/disabling a plugin adds/removes its templates. - sdk: WorkflowTemplate type + optional `templates` on IntegrationPackage - plugin-github: src/templates.ts (githubTemplates, the code-review template) threaded into the package export next to the github.* actions it uses - registry: listTemplates() flattens every plugin's contributed templates - worker: workflow-templates.ts reads from the registry (install path unchanged) - GET /api/templates gates on org_plugins — a plugin disabled org-wide drops its templates from the gallery No codegen change needed: generate-plugin-registry imports each plugin's default export, so the new `templates` field rides along automatically.
…t publish 'anthropic/claude-sonnet-4-6' is rejected by parseModelId (expects 'anthropic:<model>'), so every code-review template install 500'd with llm_model_id_invalid. Caught in live local testing; the template validation test didn't catch it because it runs without a configured provider key.
The landing hides the tab strip (full-bleed hero), and its card grid predates the gallery — so first click on Automation offered no path to Templates. Add a Templates NavCard (live count) as the first card and retitle the section Inspect -> Explore since Templates is a starting point, not an inspection target.
- Install route translates WorkflowVersionError to 400 (503+Retry-After
for publish_contention), mirroring the /api/workflows publish route —
a template whose model/provider isn't configured is a caller-fixable
error, not a 500. Route headers use the /** METHOD /path */ style.
- 'Run now' inputs derive from the trigger node's dataSchema
(templateRunInputs) instead of a duplicate WorkflowTemplateInput list
on the template — WorkflowInputDefinition gains advisory label/
placeholder; wire shape and client untouched.
- plugin-github templates.ts moves into src/actions/ with the rest of
the ./actions export surface; WorkflowTemplate.apps doc now covers
non-service brand tokens ('claude').
- Rollback failure during install logs a console.warn instead of a
silent swallow.
- Tests: publish-time env validation over every template (the gate that
would have caught the slash-form model id), plus installWorkflowTemplate
coverage against a real migrated SQLite — happy path, env-gate rollback,
repeat-install collision, unknown id.
SearchInput + Skeleton + cn() instead of hand-rolled equivalents; an error state distinct from the empty state (a failed /api/templates no longer renders as 'No matching templates'); clipboard copy wrapped in try/catch with a fallback toast (webhook-token-reveal idiom); reuse useRunWorkflow (restoring clientRequestId dedup + Runs invalidation) instead of a duplicate hook; templateKeys gains the list() leaf.
dag/v1 stores node.action as the full service-prefixed id
('github.inspect_pull_request'), so composing service + action rendered
'github.github.inspect_pull_request' in trace cards, canvas subtitles,
approval prompts, and the disabled-action error. Pre-existing, surfaced
by this PR's template; guarded with startsWith so hand-typed unprefixed
custom actions keep the composed form.
c20945f to
ada311e
Compare
|
Solid draft overall — architecture is coherent and the intent is delivered. A few issues worth addressing before merge. Intent check: The diff delivers everything described: Findings:
|
…tions prompt The code-review template now targets a REPO, not one PR: the webhook maps the GitHub event `action`, and an `if` gate only runs the review on events that introduce or change code (opened/reopened/synchronize/ready_for_review). Noise events (closed, labeled, assigned, …) short-circuit before the LLM call — verified live: a 'closed' delivery completes in ~40ms with fetch/ review/post skipped, an 'opened' delivery runs the full pipeline. Manual 'Run now' still works via the gate's isEmpty arm. The review prompt now judges on three axes: intent-vs-changeset (does the diff do what the title/body claim?), conventions (consistency with patterns visible in the surrounding diff + sibling files), and correctness. It leads with an explicit Intent check. `action` is a new hidden trigger input — part of the invocation contract so trigger-data validation accepts it on webhooks, but WorkflowInputDefinition gains an advisory `hidden` flag so templateRunInputs keeps it out of the 'Run now' form.
The App manifest registered deliveries to /api/webhooks/github, which is behind the /api/* auth middleware (no webhook exclusion) — so every App-delivered event 401'd. The handler is mounted at /webhooks/github (before the auth middleware). One-line fix; already-created Apps still need their webhook URL hand-edited in GitHub settings.
Tool nodes gain an opt-in credential:'app' — resolves the org GitHub App installation (bot) token instead of the workflow owner's personal one, degrading gracefully to the owner's token where no App is installed. The code-review template's post node uses it, so reviews post as the bot when the App is set up. create_comment now appends the app-authored attribution footer under a bot token (per the AI-labeling requirement).
A new 'github-app' trigger (scoped to owner/repo) fires from the org GitHub App event stream — no per-workflow webhook to configure. A pull_request delivery fans out to every matching enabled trigger, mapping the payload into trigger.data via the trigger's variableMapping, deduped per delivery id, filtered by subscribed event. Migration 0024 adds 'github-app' to the triggers.type CHECK (table recreate).
The setup dialog now leads with 'Arm it for a repository': pick a connected repo, and if the Valet App covers its owner, one click creates a github-app trigger (POST /api/templates/:id/enable-app) — reviews every PR, posted as the bot, no webhook paste. The manual webhook is demoted to a collapsible fallback. Coverage + ownership validated server-side; dialog state now resets on close.
…ence The runForm hint (WorkflowTemplate + WorkflowTemplateSummary) was added to the client/worker consumers but its type declarations were never committed — the branch typechecked only against a locally-built dist. Commit the source so CI (which builds from source) stays green.
The zero-param vi.fn made the mock's call tuple empty, so indexing [1] failed the workspace typecheck (tsc --build, stricter than the worker's tsc --noEmit). Type the mock params and cast the asserted input.
Re-arming a repo already armed for the user hit the per-user unique trigger-name index (idx_triggers_user_name) and leaked a raw D1 constraint error as a 500 'Internal server error'. enableTemplateGithubApp now checks for an existing github-app trigger with the same name first and returns it with alreadyArmed:true instead of creating a duplicate. The dialog shows 'Already armed' in that case.
…dialog Drop the 'Template added' toast that fired on install (it popped the moment the arming section appeared). Rename the user-facing 'Arm it for a repository' / 'Armed for …' copy to 'Install …', plus the internal section component/state, per product wording.
…he button Picking a repo that already has a github-app trigger now shows the '✓ Installed' confirmation instead of the 'Enable via GitHub App' button, so you can't double-install. Derived from the triggers list (client Trigger type + config gain the github-app variant); combined with the just- installed local state so the confirmation persists after enabling.
… failure tryApp is now a total function — a throw from installation-token minting (e.g. a misconfigured App / undecodable JWT) returns ok:false instead of propagating, so credentialMode:'app' degrades to the caller's own token rather than failing the workflow's post step.
…nly, @valet re-review) decideReview gates App-driven reviews: never review a draft PR; post the initial review on opened/reopened/ready_for_review; do NOT re-review on a push (synchronize); re-review only when a PR comment @-mentions Valet (with a bot-comment loop guard). github-app triggers now subscribe to pull_request AND issue_comment, and the webhook route dispatches both. The dispatch sends clean {owner,repo,pullNumber} (no action) since the decision is made up front — the template gate passes via its isEmpty arm.
|
Solid PR overall — the architecture is clean, the atomicity story for install is well-handled, and the test coverage is meaningfully hermetic. A few real issues worth acting on:
|
…ush-review policy Review prompt now optimizes for signal: report only issues that matter, say so briefly when a PR is solid, and — critically — do not claim something is missing (auth, a guard, a test) just because it is absent from the diff (it likely lives elsewhere in the repo). Also drop 'synchronize' from the gate so the raw-webhook path matches decideReview: review on open/reopen/ready, re-review only on @valet.
…iew, not 'valet' The generic word 'valet' resolves to a real, unrelated GitHub user (github.com/valet), so '@valet' in a comment pinged a stranger and wasn't the bot. decideReview now matches the installed App's own handle — @<slug> or the collision-proof @<slug>[bot] login (e.g. @valet-turnkey) — resolved from the App config; with no configured slug it never matches.
The code-review template's @-mention re-review reacts to issue_comment webhooks, but the manifest only requested push + pull_request — so a freshly-created App never received comment events. Add issue_comment to the default manifest events.
…allback Per review: an automation that posts as the bot must never silently fall back to a person's token. credentialMode='app' now resolves the App installation (anonymous/bot) credential and, if unavailable, fails with an actionable 'ask an admin to install the App' error instead of using the caller's personal token. Default (non-app) mode is unchanged.
…action policies Add admin (org) and per-user controls for the AI code-review automation, with an org-ceiling + user-may-only-loosen precedence that mirrors resolveEffectiveActionPolicy — the org value is a hard ceiling and a user may only make review quieter for their own repos, never override an org OFF. Policy-shaped controls (resolved by resolveCodeReviewGate): - org: codeReviewEnabled (master, OFF is absolute), codeReviewEnforced (locks the user knobs, the userGrantBehavior:'blocked' analog), codeReviewMentionOnly (org-wide quiet mode) — GitHubServiceMetadata, via PUT /api/admin/github/settings. - user: codeReviewEnabled (owner opt-out) + codeReviewMentionOnly (owner quiet mode) — users.code_review_* (migration 0025), via PATCH /api/auth/me. Enforcement in dispatchGithubAppReviews: org gate short-circuits up front; the per-owner gate reads the arming user's prefs (skipped entirely when the org enforces). trigger.user_id is the repo owner who armed the automation. UI: org toggles in the admin GitHub settings panel; a user "Code review" section under Settings > Agent. Prompt/model tuning are deliberately left as a future non-policy seam (they have no order relation, so they don't fit a mode/deny/allow resolver). Tests: resolveCodeReviewGate truth tables + dispatch org/owner gate coverage.
…ered repo When a picked repo's owner has no App installation, offer a direct 'Install the Valet App on <owner>' button (github.com/apps/<slug>/installations/new) instead of a dead-end 'ask an admin' message — installing the App is part of arming the automation. Falls back to a plain one-line notice when the App isn't configured at all.
Self-review pass to trim overengineering and match repo conventions; no behavior change (worker 1250/1250, client 174/174, tsc clean). - Fix comments/docstrings that contradicted the code: the github resolver docstring + tryApp comment and the template post-node comment all claimed credential:'app' falls back to the user token (it never does); the template header listed 'synchronize' as a gate arm (the gate deliberately omits it). - Remove dead isBotToken ternary in github.create_comment — every sibling appends attributionSuffix(ctx) unconditionally (it's '' without attribution). - Delete the up-front structural validation in installWorkflowTemplate — it duplicated the publishDraft gate byte-for-byte and the rollback already covers the impossible bad-template case. - Drop the never-rendered WorkflowTemplateInput.description field. - Collapse the 12-line precedence comment on GitHubServiceMetadata (the rule lives on resolveCodeReviewGate) and relocate the orphaned dispatch JSDoc. - DRY: qualifyActionId() helper (lib/action-id.ts) for the service-prefix rule inlined in tool.ts + approval-view.ts; export toolCallName() from the editor model instead of a second copy in trace-node-card; a shared RepoSelect for the arm + Run-now forms.
Templates gallery + PR code-review automation
A Zapier-style Templates gallery on top of the
dag/v1workflow interpreter (#43), plus the first template: an automated PR code-review workflow driven by the org's GitHub App. Installing a template creates a real, published, runnable workflow via the canonicalcreate → publish → triggerpath — no bespoke execution engine — and the code-review template can be armed on a repo with one click, no manual webhook setup.What's included
GET /api/templatesfilters by theorg_pluginsswitch, so disabling a plugin removes its templates.github-apptrigger type: one App delivery fans out to every matching repo trigger. Claude reviews the PR diff on three axes — intent vs. changeset, conventions, correctness — and posts a single review comment, leading with an intent check.codeReviewEnabled / enforced / mentionOnlyplus per-owner prefs, precedence modeled on the action-policy system (admin GitHub settings + Settings → Agent).credential:'app'posts as the org bot, never as a person.How it works
/webhooks/githubreceives App deliveries;dispatchGithubAppReviewsreads the org policy once (an org OFF is absolute), then per armed trigger evaluatesresolveCodeReviewGate(org, owner, reason). Precedence is an org ceiling — the owner may only loosen, never tighten past the org or override an org OFF (the same shape asresolveEffectiveActionPolicy).decideReviewapplies a Greptile-style policy: skip drafts/pushes, review on open/reopen/ready, re-review only on an@<bot>mention. Dispatch is best-effort — one trigger failure never blocks the webhook 200.The review posts under the App-installation (bot) token:
credential:'app'resolves the bot token only and fails with an actionable "install the App" message if it's unavailable — it does not silently fall back to the caller's personal token.github.inspect_pull_requestgains an opt-inincludePatch(backward-compatible) so the reviewer sees the actual diff.Two run paths: an armed App trigger (automatic, every PR) and manual "Run now" with
{ owner, repo, pullNumber }via the gate'sisEmptyarm. A manual per-repo webhook remains as a demoted fallback.Testing
Full local end-to-end: install via API and gallery UI, then both trigger paths through all nodes to a real generated review (which correctly caught injected bugs). Unit/integration coverage for the credential resolver (app vs. default), the code-review gate matrix, dispatch-level no-dispatch paths (draft/push, generic
@valet, wrong repo, disabled trigger, org gates), and template install (structural validity, publish, trigger creation, repeat-install collision).tscclean across worker/shared/client; worker 1250, client 174 vitest green.Follow-ups
action_policiesframework proper (org rows + user allow-rows) is a natural next step — the settings were shaped to migrate cleanly.