From d27934ebca4d84708666e05bdc8b01a25ac0b58d Mon Sep 17 00:00:00 2001 From: pengfei-threemoonslab Date: Fri, 22 May 2026 17:46:00 -0700 Subject: [PATCH 1/2] Add advisory CI design partner pilot runbook --- README.md | 9 +- docs/INDEX.md | 1 + docs/agents/use-with-codex.md | 8 +- docs/design-partner-advisory-ci-pilot.md | 152 +++++++++++++++++++++++ docs/design-partners.md | 16 ++- docs/target-repo-agent-snippets.md | 5 +- 6 files changed, 183 insertions(+), 8 deletions(-) create mode 100644 docs/design-partner-advisory-ci-pilot.md diff --git a/README.md b/README.md index a6aea80c..ed5247d5 100644 --- a/README.md +++ b/README.md @@ -110,7 +110,7 @@ Run: agents-shipgate detect --workspace . --json If Shipgate is relevant, run: agents-shipgate contract --json -agents-shipgate init --workspace . --write --ci --json +agents-shipgate init --workspace . --write --ci --json --agent-instructions=all agents-shipgate scan -c shipgate.yaml --suggest-patches --format json agents-shipgate apply-patches --from agents-shipgate-reports/report.json --confidence high --apply Read `agents-shipgate-reports/report.json`, not Markdown. Summarize @@ -127,7 +127,7 @@ For OpenAI Codex repos, install both the native `AGENTS.md` trigger block and the repo-scoped Codex skill: ```bash -agents-shipgate init --workspace . --write --agent-instructions=agents-md,codex-skill +agents-shipgate init --workspace . --write --ci --json --agent-instructions=agents-md,codex-skill ``` The skill lives at `.agents/skills/agents-shipgate/`, can be invoked with @@ -502,7 +502,9 @@ Teams shipping production-like tool-using agents can apply to the [Three Moons Lab design partner program](https://threemoonslab.com/design-partners/) — the marketing page mirrors [`docs/design-partners.md`](docs/design-partners.md) in the repo and includes a -prefilled email CTA for review criteria and contact. +prefilled email CTA for review criteria and contact. For the current +agent-installed advisory CI pilot, use +[`docs/design-partner-advisory-ci-pilot.md`](docs/design-partner-advisory-ci-pilot.md). ## Docs @@ -521,6 +523,7 @@ readers and AI search ingest. - [Check catalog](docs/checks.md) - [Policy packs](docs/policy-packs.md) - [Baseline workflow](docs/baseline.md) +- [Design partner advisory CI pilot](docs/design-partner-advisory-ci-pilot.md) - [JSON report schema v0.20](docs/report-schema.v0.20.json) - [Privacy and redaction](docs/privacy.md) - [Trust model](docs/trust-model.md) diff --git a/docs/INDEX.md b/docs/INDEX.md index e45cc424..4b63313c 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -10,6 +10,7 @@ A single entry point for human readers and AI agents walking the `docs/` tree. - [`glossary.md`](glossary.md) — category vocabulary - [`ai-search-summary.md`](ai-search-summary.md) — human-readable summary for AI search and coding agents - [`design-partners.md`](design-partners.md) — early design partner criteria and contact path +- [`design-partner-advisory-ci-pilot.md`](design-partner-advisory-ci-pilot.md) — runbook for getting design partners onto agent-installed advisory CI - [`architecture.md`](architecture.md) — codebase layout for new contributors - [`manifest-v0.1.md`](manifest-v0.1.md) — manifest schema in prose form - [`trust-model.md`](trust-model.md) — what the scanner does and doesn't do diff --git a/docs/agents/use-with-codex.md b/docs/agents/use-with-codex.md index f8f7bcd1..4b5f1e5a 100644 --- a/docs/agents/use-with-codex.md +++ b/docs/agents/use-with-codex.md @@ -18,13 +18,17 @@ From the root of the project where Codex should run Shipgate: ```bash pipx install agents-shipgate -agents-shipgate init --workspace . --write --agent-instructions=agents-md,codex-skill +agents-shipgate init --workspace . --write --ci --json --agent-instructions=agents-md,codex-skill ``` +This installs the Codex trigger block and repo-scoped skill, and writes the +first GitHub Actions workflow in advisory mode. If the repo already has +Shipgate CI, `init` refuses to overwrite it and reports the skip in JSON. + To install every supported agent surface at once: ```bash -agents-shipgate init --workspace . --write --agent-instructions=all +agents-shipgate init --workspace . --write --ci --json --agent-instructions=all ``` The `codex-skill` target writes `.agents/skills/agents-shipgate/`. It is diff --git a/docs/design-partner-advisory-ci-pilot.md b/docs/design-partner-advisory-ci-pilot.md new file mode 100644 index 00000000..eb260b56 --- /dev/null +++ b/docs/design-partner-advisory-ci-pilot.md @@ -0,0 +1,152 @@ +# Design Partner Advisory CI Pilot + +Use this runbook to get three design partners running agent-installed +advisory CI for Agents Shipgate. + +## Goal + +Three design partner repos have Agents Shipgate advisory CI installed by a +coding agent and exercised on at least one pull request. + +## Definition Of Running + +A partner counts as running advisory CI when all of these are true: + +- The partner repo contains a reviewed `shipgate.yaml` with no unresolved + `CHANGE_ME` values. +- A coding agent installed or updated the Shipgate adoption surface using + `agents-shipgate init --write --ci --agent-instructions=...` or an + equivalent agent-driven workflow. +- `.github/workflows/agents-shipgate.yml` is present and uses + `ci_mode: advisory`. +- A pull request has run the workflow and produced a Shipgate PR comment or + uploaded report artifact. +- The agent or maintainer read `agents-shipgate-reports/report.json` and used + `release_decision.decision` as the release signal. +- `agents-shipgate-reports/` is ignored and not committed. + +## Partner Fit + +Prioritize teams that can complete a 30-45 minute first install and share +actionability feedback within one week. + +Good first partners usually have: + +- A GitHub repo with an agent that calls tools through MCP, OpenAPI, OpenAI + Agents SDK, Anthropic Messages API, Google ADK, LangChain/LangGraph, CrewAI, + OpenAI API artifacts, Codex plugin packages, or n8n workflow JSON. +- At least one write, customer-communication, financial, destructive, sensitive + read, deployment, or record-modifying tool. +- A coding agent already used for PR work, such as Codex, Claude Code, Cursor, + Aider, or a similar local assistant. +- Permission to run advisory CI that comments on pull requests but does not + block merges. +- A named reviewer who can judge whether the findings are useful. + +Avoid first-wave partners that need hosted dashboards, runtime enforcement, +private-data upload, compliance certification, or non-GitHub CI as the primary +success path. + +## Pilot Path + +1. Qualify the repo and confirm the agent framework, tool sources, CI system, + and coding agent. +2. Ask the partner to create a branch for the Shipgate install. +3. Have the partner paste the agent prompt below into their coding agent. +4. Review the generated `shipgate.yaml`, workflow, and first report. +5. Open or update a pull request and wait for the advisory workflow. +6. Capture the workflow URL, PR comment URL, top findings, false positives, and + install friction. +7. Schedule a 20-minute readout within one week. + +## Partner Agent Prompt + +Paste this into the partner's coding agent from the root of the target repo: + +```text +Add Agents Shipgate as an advisory Tool-Use Readiness release gate for this +tool-using AI agent repo. + +Use the agent-installed path: +1. Install agents-shipgate if it is missing: + pipx install agents-shipgate + If pipx is unavailable, use python -m pip install agents-shipgate. +2. Set AGENTS_SHIPGATE_AGENT_MODE=1 and run: + agents-shipgate detect --workspace . --json + agents-shipgate contract --json + agents-shipgate init --workspace . --write --ci --json --agent-instructions=all +3. Replace every CHANGE_ME value in shipgate.yaml using the repo's prompt, + README, or main agent file. +4. Run: + agents-shipgate scan -c shipgate.yaml --suggest-patches --format json --ci-mode advisory + agents-shipgate apply-patches --from agents-shipgate-reports/report.json --confidence high --apply --json +5. Ensure agents-shipgate-reports/ is ignored and not committed. +6. Summarize release_decision.decision, blocker count, review item count, the + top three critical/high findings, safe patches applied, and anything that + needs human review. + +Do not enable strict CI, save a baseline, suppress findings, or auto-assert +approval, confirmation, idempotency, broad-scope, prohibited-action, or runtime +trace evidence. +``` + +If the partner wants the smallest Codex-only install, use +`--agent-instructions=agents-md,codex-skill` instead of `all`. For Claude Code, +use `--agent-instructions=agents-md,claude-md,claude-code-skill`. + +## First Call Agenda + +- Confirm the agent's purpose, framework, and tool source boundary. +- Confirm GitHub Actions and PR comments are acceptable for the first pass. +- Run or supervise the partner-agent prompt. +- Fill unresolved `CHANGE_ME` values from source context. +- Open the install PR and verify the first local report. +- Agree what evidence can be shared back with Three Moons Lab. + +## Success Tracker + +Keep private notes outside the public repo. A lightweight tracker row should +include: + +| Field | Value | +| --- | --- | +| Partner | | +| Repo / agent type | | +| Tool source type | | +| Coding agent | | +| Install PR | | +| Advisory workflow run | | +| PR comment or artifact | | +| `release_decision.decision` | | +| Blockers / review items | | +| Top useful finding | | +| False positive or friction | | +| Follow-up date | | + +## Follow-Up Questions + +Ask these after the first PR comment lands: + +- Did the agent discover and run Shipgate without manual command-by-command + coaching? +- Which finding would change a release decision or force a policy discussion? +- Which finding was noisy, confusing, or missing enough context? +- Did the PR comment contain the right level of detail for reviewers? +- What would make the team comfortable moving from advisory to strict later? + +## Outreach Snippet + +```text +We are looking for three design partners to try Agents Shipgate in advisory CI. +The pilot is local-first: your coding agent installs a shipgate.yaml, adds a +non-blocking GitHub Action, and opens a PR with a Tool-Use Readiness report. +No agent execution, LLM calls, MCP connections, hosted dashboard, or telemetry +are required. In return, we ask for feedback on whether the findings and PR +comment are useful for your platform/security/release reviewers. +``` + +## Exit Criteria + +The goal is met when three tracker rows satisfy the definition of running, each +has a first-run feedback note, and at least one concrete product or docs +follow-up has been captured from each partner. diff --git a/docs/design-partners.md b/docs/design-partners.md index b372bb7b..5b0169a5 100644 --- a/docs/design-partners.md +++ b/docs/design-partners.md @@ -4,6 +4,17 @@ Three Moons Lab is looking for early design partners who are shipping tool-using AI agents and want a repeatable release-readiness review before production-like permissions are granted. +## Advisory CI Pilot + +The current pilot goal is three design partner repos running +agent-installed advisory CI: a coding agent adds `shipgate.yaml`, installs the +repo-level Shipgate agent instructions, creates a non-blocking GitHub Actions +workflow, and exercises it on a pull request. + +Use the [`Design Partner Advisory CI Pilot`](design-partner-advisory-ci-pilot.md) +runbook for the qualification checklist, partner-agent prompt, success +definition, and follow-up questions. + ## Good Fit You are likely a good fit if your team: @@ -14,6 +25,7 @@ You are likely a good fit if your team: - Has tools that refund, email, cancel, deploy, modify records, read sensitive data, or change infrastructure. - Wants advisory PR evidence before moving to stricter CI behavior. +- Uses a coding agent that can install release-gate changes in a branch. - Can share sanitized findings, workflow constraints, or integration feedback with Three Moons Lab. @@ -47,4 +59,6 @@ Email `help@threemoonslab.com` with the subject `Agents Shipgate design partner review`. Include the agent framework, tool-source types, current CI system, and whether -you want a local CLI workflow, a GitHub Action workflow, or both. +you want a local CLI workflow, a GitHub Action workflow, or both. If you want +the fastest pilot path, include which coding agent you use and whether it can +open a branch or pull request in your repo. diff --git a/docs/target-repo-agent-snippets.md b/docs/target-repo-agent-snippets.md index 19ff4473..3ef91b61 100644 --- a/docs/target-repo-agent-snippets.md +++ b/docs/target-repo-agent-snippets.md @@ -5,9 +5,10 @@ Agents Shipgate visible to coding agents working in that target repo, not only to agents reading the Agents Shipgate source repo. > The CLI plants these snippets for you. Run -> `agents-shipgate init --write --agent-instructions=all` (or pass a subset +> `agents-shipgate init --write --ci --agent-instructions=all` (or pass a subset > like `--agent-instructions=agents-md,codex-skill,claude-code-skill,cursor`) -> to emit them into the target repo. Shared host files use managed +> to emit them into the target repo and add the first advisory GitHub Actions +> workflow. Shared host files use managed > `` blocks; full-file and skill-bundle targets > use safe-update checks. Idempotent — safe to rerun. The raw content below is > the canonical reference and the source the renderers in From 1c5b28853144bb32e3a0581984cf68e5f7893269 Mon Sep 17 00:00:00 2001 From: pengfei-threemoonslab Date: Fri, 22 May 2026 17:58:29 -0700 Subject: [PATCH 2/2] Clarify advisory CI pilot adoption path --- docs/design-partner-advisory-ci-pilot.md | 26 ++++++++++++++---------- docs/target-repo-agent-snippets.md | 12 +++++------ 2 files changed, 21 insertions(+), 17 deletions(-) diff --git a/docs/design-partner-advisory-ci-pilot.md b/docs/design-partner-advisory-ci-pilot.md index eb260b56..d79ce37c 100644 --- a/docs/design-partner-advisory-ci-pilot.md +++ b/docs/design-partner-advisory-ci-pilot.md @@ -15,8 +15,12 @@ A partner counts as running advisory CI when all of these are true: - The partner repo contains a reviewed `shipgate.yaml` with no unresolved `CHANGE_ME` values. - A coding agent installed or updated the Shipgate adoption surface using - `agents-shipgate init --write --ci --agent-instructions=...` or an - equivalent agent-driven workflow. + `agents-shipgate init --write --ci --agent-instructions=...` or a workflow + that explicitly runs that same init step. + `agents-shipgate bootstrap` alone is not sufficient for this pilot because + it does not install the agent-instruction surfaces; run + `init --write --ci --agent-instructions=...` after bootstrap or use that + `init` command instead. - `.github/workflows/agents-shipgate.yml` is present and uses `ci_mode: advisory`. - A pull request has run the workflow and produced a Shipgate PR comment or @@ -27,14 +31,12 @@ A partner counts as running advisory CI when all of these are true: ## Partner Fit -Prioritize teams that can complete a 30-45 minute first install and share -actionability feedback within one week. +Use the general fit criteria in [`design-partners.md`](design-partners.md). +For this pilot, expect 30-45 minutes for the first install and prioritize +teams that can share actionability feedback within one week. Good first partners usually have: -- A GitHub repo with an agent that calls tools through MCP, OpenAPI, OpenAI - Agents SDK, Anthropic Messages API, Google ADK, LangChain/LangGraph, CrewAI, - OpenAI API artifacts, Codex plugin packages, or n8n workflow JSON. - At least one write, customer-communication, financial, destructive, sensitive read, deployment, or record-modifying tool. - A coding agent already used for PR work, such as Codex, Claude Code, Cursor, @@ -75,8 +77,9 @@ Use the agent-installed path: agents-shipgate detect --workspace . --json agents-shipgate contract --json agents-shipgate init --workspace . --write --ci --json --agent-instructions=all -3. Replace every CHANGE_ME value in shipgate.yaml using the repo's prompt, - README, or main agent file. +3. Replace every CHANGE_ME value in shipgate.yaml using the agent's system + prompt file, README, or main agent module. This pilot assumes the default + auto-detected init template, not --minimal. 4. Run: agents-shipgate scan -c shipgate.yaml --suggest-patches --format json --ci-mode advisory agents-shipgate apply-patches --from agents-shipgate-reports/report.json --confidence high --apply --json @@ -105,8 +108,9 @@ use `--agent-instructions=agents-md,claude-md,claude-code-skill`. ## Success Tracker -Keep private notes outside the public repo. A lightweight tracker row should -include: +Keep private notes outside the public repo. + +Template: copy into your private tracker: | Field | Value | | --- | --- | diff --git a/docs/target-repo-agent-snippets.md b/docs/target-repo-agent-snippets.md index 3ef91b61..fc022f61 100644 --- a/docs/target-repo-agent-snippets.md +++ b/docs/target-repo-agent-snippets.md @@ -5,7 +5,7 @@ Agents Shipgate visible to coding agents working in that target repo, not only to agents reading the Agents Shipgate source repo. > The CLI plants these snippets for you. Run -> `agents-shipgate init --write --ci --agent-instructions=all` (or pass a subset +> `agents-shipgate init --write --ci --json --agent-instructions=all` (or pass a subset > like `--agent-instructions=agents-md,codex-skill,claude-code-skill,cursor`) > to emit them into the target repo and add the first advisory GitHub Actions > workflow. Shared host files use managed @@ -49,7 +49,7 @@ Commands: ```bash agents-shipgate detect --workspace . --json agents-shipgate contract --json -agents-shipgate init --workspace . --write --ci --json +agents-shipgate init --workspace . --write --ci --json --agent-instructions=all agents-shipgate scan -c shipgate.yaml --suggest-patches --format json agents-shipgate apply-patches \ --from agents-shipgate-reports/report.json \ @@ -76,13 +76,13 @@ For OpenAI Codex, generate the repo-scoped skill into `.agents/skills/agents-shipgate/`: ```bash -agents-shipgate init --workspace . --write --agent-instructions=codex-skill +agents-shipgate init --workspace . --write --ci --json --agent-instructions=codex-skill ``` Pair it with the `AGENTS.md` block for the strongest trigger surface: ```bash -agents-shipgate init --workspace . --write --agent-instructions=agents-md,codex-skill +agents-shipgate init --workspace . --write --ci --json --agent-instructions=agents-md,codex-skill ``` The skill can be invoked explicitly with `$agents-shipgate` and may be used @@ -96,14 +96,14 @@ For Claude Code, generate the repo-scoped skill into `.claude/skills/agents-shipgate/`: ```bash -agents-shipgate init --workspace . --write --agent-instructions=claude-code-skill +agents-shipgate init --workspace . --write --ci --json --agent-instructions=claude-code-skill ``` Pair it with the `AGENTS.md` block and the `CLAUDE.md` managed-block for the strongest trigger surface: ```bash -agents-shipgate init --workspace . --write \ +agents-shipgate init --workspace . --write --ci --json \ --agent-instructions=agents-md,claude-md,claude-code-skill ```