feat(automated-ai-engineer): add the agent-improver meta-engineer agent#73
Conversation
The plugin ships the engineer role but nothing that improves the engineer from the outside. Its `self-improvement` skill is one run reflecting on its own memory — so it cannot see failures recurring across hundreds of runs, waste that looks normal from inside a run, divergence between sibling instances, or drift between a non-version-controlled bootstrap entry and the contract it points at. Adds `agent-improver`: an external observer over the whole session corpus and every deployed instance, scoring the engineer on reliability, safety, efficiency, quality, coordination and currency, and shipping evidence-backed, reversible fixes to its definition. Two parts carry most of the weight. The ingestion boundary — a meta-engineer with authority over the definition is the deployment's highest-value injection target, so it mines BEHAVIOUR and never treats corpus prose as instruction. And distinguishing "the guard is wrong" from "the agent is wrong", which look identical in telemetry and whose confusion is how a working control gets deleted to silence a symptom. The agent's obligations (evidence, reversibility, audit trail, loosenings ship alone) hold under any authority model, since deployments differ in how much a meta-engineer may change alone. The matching `agent-improvement` skill bundle follows once its upstream merges (devantler-tech/agent-skills#71) — bundled skills must carry `gh skill install` provenance and are never hand-copied. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2c6e352def
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex review on #73 found the new agent's requirements were not reflected in the consumer docs. P2: enabling agent-improver makes two more AGENTS.md sections load-bearing — Agent definition locations (what it may change, and which surfaces are version-controlled vs edited-in-place) and Authority model (how much it may change alone, split by tighten/loosen and prose/enforcement). A consumer following the documented five-section setup would have neither, so the meta-engineer could not run safely. Also notes what Memory must hold for the verification loop. P3: the root README's custom-agent delivery section still described two bundled agents and gave VS Code copy targets for only those two, so anyone following the manual path would not discover the third. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Findings addressed and pushed — re-requesting review at the new head. @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 71cbf84a5c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Closes the merge gate on this PR. The agent-improver agent declares and delegates to agent-improvement, so publishing the agent without the bundle would ship an unresolved skill reference (Codex P1 on this PR). Installed via `gh skill install` from the upstream that just merged (devantler-tech/agent-skills#71), so it carries the metadata.github-* provenance the manifest validator requires — a hand-copied SKILL.md is rejected, correctly. Verified the bundled body is byte-identical to upstream main. README resource index updated to match on-disk resources, which the validator enforces. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Merge gate cleared in 336ae97. README resource index updated to match (the validator caught that too). 55 manifest checks pass. @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 336ae97084
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ng sections Four review findings, three of them the same shape: adding a third agent and a fourth skill left every count in the docs stale, each in a different file. - Root README claimed npx resolves "all 27 bundled skills"; the tree now holds 28. - Plugin README's Delivery section still said "both bundled agents" one screen after introducing the third. - ADR 0002 still described the bundle as "two agents + three skills" while being cited as its design record. Updated to note the extension rather than rewriting history. The substantive one: the two AGENTS.md sections this agent depends on (Agent definition locations, Authority model) were named only in the frontmatter description, so the prompt body never told the model to read them or to stop when they are absent — unlike the sibling agent, which fails closed per dimension. Added the same contract, with the reason stated: those sections are what constrain a role that edits its own guardrails, so an agent that improvises them has removed its own limits before doing anything else. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All review findings addressed in @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1b315d9dd1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…uctions My previous fix corrected the agent COUNT but kept the claim that all agents load automatically everywhere. Under this repo's documented delivery model (root README, Custom agents) VS Code consumes agents but does not bundle them from a plugin — they must be copied into .github/agents/*.agent.md. So a VS Code user adding the new agent-improver would have followed this page, skipped the copy, and found the agent simply absent with nothing to explain it. Now states the split explicitly and lists all three copy targets. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All three valid. Two of them could not be fixed here — which is the third finding proving itself. VS Code agent delivery — fixed in Contract checks + vendored routing — these live in the bundled Your third finding is the more interesting of the two, because the skill was telling itself to do the wrong thing: it treated skills as a version-controlled surface to patch in place, so a future improver run finding a defect in a bundled skill would have edited the copy, watched the next sync revert it, and concluded its own fix had failed. Now generalised — check for an origin marker before editing any definition file you did not author. The second is the one with teeth: the skill is installable standalone, so without the wrapper nothing checked Agent definition locations or Authority model before step 4 routed edits at bootstrap entries and permission files. A skill that edits guardrails has to establish its own bounds. It now fails closed per dimension. Merge order: this PR waits on agent-skills#72, then re-bundles via |
…red fixes upstream (#72) Two findings from the Codex review of the plugin that bundles this skill (devantler-tech/agent-plugins#73). Contract guard: the skill is installable and invocable on its own, without the agent-improver wrapper that checks the deployment's Agent definition locations and Authority model. The pre-flight only checked that some definition files exist, then step 4 happily routed edits at bootstrap entries and permission configuration. A skill that edits guardrails must establish its own bounds, so it now performs those checks itself and fails closed per dimension. Vendored copies: the routing table treated "skills" as a version-controlled surface to patch in place. Where a definition file was installed from another repository — a bundled SKILL.md, a vendored agent — the consuming repo generally forbids hand-editing it and the next sync reverts the change, so the defect returns and the fix appears to have failed. Now says to fix it at its upstream and check for an origin marker before editing any definition file you did not author. Both are stated generically rather than in marketplace terms, since the skill is deployment-neutral; the plugin case is one instance of the pattern. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Picks up devantler-tech/agent-skills#72 — the standalone contract guard and the rule routing vendored-copy fixes to their upstream. Installed with `gh skill install`, so provenance advances v1.9.0 -> v1.9.1 and the body is verified byte-identical to upstream main. This is the path the skill's own new rule prescribes: the previous two findings could not be fixed in this copy, because the next sync would have reverted them. Note for the next re-bundle: `gh skill install` resolves to the latest RELEASE tag, not main HEAD, so it must run after the release workflow finishes — an install fired seconds after merge silently returns the previous version. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Upstream merged and re-bundled in Worth recording: Re-requesting review at the new head. @codex review |
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Why
The plugin ships the autonomous engineer role, but nothing that improves the engineer from the outside. Its
self-improvementskill is one run reflecting on its own memory — so it cannot see the failures that only appear in aggregate: errors recurring across hundreds of runs that look like one-offs from inside any single one, waste that feels normal in the moment, divergence between sibling instances, and drift between a bootstrap entry (usually not version-controlled) and the contract it points at.Run against a live deployment, this agent found four such defects on its first pass — including two loaders still instructing agents to follow a rule retired that morning.
What
Adds a third agent,
agent-improver: an external observer over the whole session corpus and every deployed instance at once. It scores the engineer on reliability, safety, efficiency, quality, coordination and currency, and ships evidence-backed, reversible fixes to its definition.Its obligations — evidence, reversibility, an audit trail, and loosenings shipping alone — hold under any authority model, since deployments differ in how much a meta-engineer may change on its own.
Consumer docs now cover the two extra
AGENTS.mdsections this agent makes load-bearing (Agent definition locations, Authority model), and the manual VS Code delivery path lists all three agents.This PR must not merge until
agent-improvementis bundled into it. The agent declares that skill and delegates its run loop to it, so merging first would publish an agent with an unresolved reference (Codex P1 on this PR — a fair catch against my original plan of a follow-up PR).The bundle cannot be hand-copied:
validate-manifests.shrequiresgh skill installprovenance, which is a rule worth keeping. So the order is:gh skill install devantler-tech/agent-skills agent-improvementinto this branch;