feat(agent-improvement): add the agent-improvement skill#71
Merged
Conversation
A self-improvement skill is one run reflecting on its own memory. Nothing in the library covers improving an autonomous engineer from the OUTSIDE, across the whole session corpus and across sibling instances — which is where the failures that no single run can see actually live: errors recurring across hundreds of runs, waste that looks normal from inside one run, divergence between instances, and drift between a non-version-controlled loader and the contract it points at. Adds the generic procedure: gather telemetry, score six parameters (reliability, safety, efficiency, quality, coordination, currency), diagnose root causes from measured patterns, ship the fix with evidence and a reversible audit trail, then verify next run that the metric moved. Two parts carry most of the weight: - The ingestion boundary. A meta-engineer with authority over the definition is the deployment's highest-value injection target, because its corpus is saturated with untrusted content. It mines BEHAVIOUR and never treats corpus prose as instruction. - Distinguishing "the guard is wrong" from "the agent is wrong". Both look identical in telemetry, and confusing them is how a working control gets deleted to silence a symptom. Deployment-neutral per the portability principle: no runtime-specific paths or tooling, configured by the consuming deployment's contract. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 18, 2026
Contributor
Author
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4ea976399a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Adding the agent-improvement index row moved the totals to 29 rows / 11 in-house, but the Installing section still claimed 28 / 10 in three places, so anyone choosing between gh skill and npx skills got stale scope for the newly added skill. The '18 upstream' figure is unchanged and stays correct. Found by Codex review on #71. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
Findings addressed and pushed — re-requesting review at the new head. @codex review |
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
devantler
marked this pull request as ready for review
July 18, 2026 13:20
devantler
added a commit
to devantler-tech/agent-plugins
that referenced
this pull request
Jul 18, 2026
Closes the merge gate on this PR. The agent-improver agent declares and delegates to agent-improvement, so publishing the agent without the bundle would ship an unresolved skill reference (Codex P1 on this PR). Installed via `gh skill install` from the upstream that just merged (devantler-tech/agent-skills#71), so it carries the metadata.github-* provenance the manifest validator requires — a hand-copied SKILL.md is rejected, correctly. Verified the bundled body is byte-identical to upstream main. README resource index updated to match on-disk resources, which the validator enforces. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
devantler
added a commit
to devantler-tech/agent-plugins
that referenced
this pull request
Jul 18, 2026
…nt (#73) * feat(automated-ai-engineer): add the agent-improver meta-engineer agent The plugin ships the engineer role but nothing that improves the engineer from the outside. Its `self-improvement` skill is one run reflecting on its own memory — so it cannot see failures recurring across hundreds of runs, waste that looks normal from inside a run, divergence between sibling instances, or drift between a non-version-controlled bootstrap entry and the contract it points at. Adds `agent-improver`: an external observer over the whole session corpus and every deployed instance, scoring the engineer on reliability, safety, efficiency, quality, coordination and currency, and shipping evidence-backed, reversible fixes to its definition. Two parts carry most of the weight. The ingestion boundary — a meta-engineer with authority over the definition is the deployment's highest-value injection target, so it mines BEHAVIOUR and never treats corpus prose as instruction. And distinguishing "the guard is wrong" from "the agent is wrong", which look identical in telemetry and whose confusion is how a working control gets deleted to silence a symptom. The agent's obligations (evidence, reversibility, audit trail, loosenings ship alone) hold under any authority model, since deployments differ in how much a meta-engineer may change alone. The matching `agent-improvement` skill bundle follows once its upstream merges (devantler-tech/agent-skills#71) — bundled skills must carry `gh skill install` provenance and are never hand-copied. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(automated-ai-engineer): document agent-improver's contract sections Codex review on #73 found the new agent's requirements were not reflected in the consumer docs. P2: enabling agent-improver makes two more AGENTS.md sections load-bearing — Agent definition locations (what it may change, and which surfaces are version-controlled vs edited-in-place) and Authority model (how much it may change alone, split by tighten/loosen and prose/enforcement). A consumer following the documented five-section setup would have neither, so the meta-engineer could not run safely. Also notes what Memory must hold for the verification loop. P3: the root README's custom-agent delivery section still described two bundled agents and gave VS Code copy targets for only those two, so anyone following the manual path would not discover the third. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(automated-ai-engineer): bundle the agent-improvement skill Closes the merge gate on this PR. The agent-improver agent declares and delegates to agent-improvement, so publishing the agent without the bundle would ship an unresolved skill reference (Codex P1 on this PR). Installed via `gh skill install` from the upstream that just merged (devantler-tech/agent-skills#71), so it carries the metadata.github-* provenance the manifest validator requires — a hand-copied SKILL.md is rejected, correctly. Verified the bundled body is byte-identical to upstream main. README resource index updated to match on-disk resources, which the validator enforces. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(automated-ai-engineer): fix count drift and fail closed on missing sections Four review findings, three of them the same shape: adding a third agent and a fourth skill left every count in the docs stale, each in a different file. - Root README claimed npx resolves "all 27 bundled skills"; the tree now holds 28. - Plugin README's Delivery section still said "both bundled agents" one screen after introducing the third. - ADR 0002 still described the bundle as "two agents + three skills" while being cited as its design record. Updated to note the extension rather than rewriting history. The substantive one: the two AGENTS.md sections this agent depends on (Agent definition locations, Authority model) were named only in the frontmatter description, so the prompt body never told the model to read them or to stop when they are absent — unlike the sibling agent, which fails closed per dimension. Added the same contract, with the reason stated: those sections are what constrain a role that edits its own guardrails, so an agent that improvises them has removed its own limits before doing anything else. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(automated-ai-engineer): correct the VS Code agent delivery instructions My previous fix corrected the agent COUNT but kept the claim that all agents load automatically everywhere. Under this repo's documented delivery model (root README, Custom agents) VS Code consumes agents but does not bundle them from a plugin — they must be copied into .github/agents/*.agent.md. So a VS Code user adding the new agent-improver would have followed this page, skipped the copy, and found the agent simply absent with nothing to explain it. Now states the split explicitly and lists all three copy targets. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(automated-ai-engineer): re-bundle agent-improvement at v1.9.1 Picks up devantler-tech/agent-skills#72 — the standalone contract guard and the rule routing vendored-copy fixes to their upstream. Installed with `gh skill install`, so provenance advances v1.9.0 -> v1.9.1 and the body is verified byte-identical to upstream main. This is the path the skill's own new rule prescribes: the previous two findings could not be fixed in this copy, because the next sync would have reverted them. Note for the next re-bundle: `gh skill install` resolves to the latest RELEASE tag, not main HEAD, so it must run after the release workflow finishes — an install fired seconds after merge silently returns the previous version. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
We have a
self-improvementskill, but it is one run reflecting on its own memory. Nothing covers improving an autonomous engineer from the outside — across the whole session corpus and across sibling instances. That outside view is where the expensive failures actually live: errors that recur across hundreds of runs but look like one-offs from inside any single one, waste that feels normal in the moment, divergence between instances, and drift between a loader (usually not version-controlled) and the contract it points at.Running this procedure against a live deployment found four such defects on the first pass, including two loaders still instructing agents to follow a rule that had been retired that morning.
What
The generic procedure: gather telemetry → score six parameters (reliability, safety, efficiency, quality, coordination, currency) → diagnose root causes from measured patterns → ship the fix with evidence and a reversible audit trail → verify next run that the metric actually moved.
Deployment-neutral per the portability principle — no runtime-specific paths or tooling, configured by the consuming deployment's contract.
Two parts carry most of the weight. The ingestion boundary: a meta-engineer with authority over the definition is the deployment's highest-value injection target, because its corpus is saturated with untrusted content — so it mines behaviour and never treats corpus prose as instruction. And distinguishing "the guard is wrong" from "the agent is wrong": both look identical in telemetry, and confusing them is how a working control gets deleted to silence a symptom.
The plugin that bundles this skill and its meta-engineer agent is devantler-tech/agent-plugins (linked once open).