Agent harness: doc map, contract locks, toolchain hardening, spec-gap flags - #2
Merged
Conversation
…dening Fixes: - README CI badges pointed at pre-rename yuque/yuque-cli - tests/e2e/README.md prose still used repo for user-facing book - group.ts/stats.ts comments contradicted cli.ts about exitOverride propagation (verified empirically: cli.ts was right); dropped the redundant .exitOverride() in group.ts Pinning: - tests/commands/flag-validation.test.ts locks exit 2 + zero HTTP for all three validation styles (choices / InvalidArgumentError / UsageError) Toolchain: - smoke:dist and test:e2e now build first; check shares one build via *:built scripts (stale-dist trap eliminated) - eslint: type-checked configs + projectService tsconfig, floating/misused promises, no-explicit-any/non-null as errors, --max-warnings 0, scripts and config files now linted; prettier covers scripts + root configs - vitest coverage thresholds (90/82/96/91) enforced in check via test:ci - smoke missing-token case clears YUQUE_TOKEN* so real dev credentials cannot leak into the smoke run Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AGENTS.md gains four sections: a Map of every src file and test directory, Contracts & locks, an 8-step Adding-a-command checklist, and Conventions & gotchas (ESM .js imports, authoritative flag-validation style, destructive-command ordering, script cheat sheet). tests/docs/agents-md.test.ts pins the guide against drift: every src/**/*.ts path must appear in the Map, critical anchors (merge gate, contract-test paths, exit-code contract line) must survive, and every `npm run <script>` mentioned must exist in package.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/docs/help-surface.test.ts serializes every leaf command (description, arguments, options with defaults and choices) plus the four operational global options into help-surface.golden.json and asserts deep equality — hallucinated or silently dropped flags now fail deterministically, closing the largest gap left by the command-name level locks. UPDATE_HELP_GOLDEN=1 regenerates the golden after a deliberate change. A reverse lock scans both READMEs for --flag tokens and requires each to exist in the registered CLI surface (one allowlisted markdown anchor). AGENTS.md's Adding-a-command checklist gains the regeneration step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/spec-constraints.test.ts extracts every enum/maximum/minimum/ required constraint from the mapped operations in spec/yuque-openapi.yaml (with local $ref resolution) and asserts bidirectional equality against a hand-reviewed CONSTRAINT_PINS table — 60 pins across 25 operations, 53 with a CLI flag, 7 explicitly cli:null. Parametrized boundary tests then drive runCli: maximum passes / maximum+1 exits 2, enum members pass with the exact wire value / outsiders exit 2, required flags fail without a value — all with zero HTTP on rejection. Deliberate CLI aliases (search --type book→repo, book list --type all→omitted) are modeled explicitly instead of skipped. Shared spec loading and OPERATION_TO_COMMANDS move to tests/utils/spec.ts, consumed by both spec-coverage and spec-constraints. Known CLI/spec gaps recorded during review (not changed here): search --offset lacks the spec's 1..100 clamp; the doc-get operations' page/ page_size query params are not exposed as flags; several CLI checks are deliberately stricter than the spec. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/docs/exit-code-docs.test.ts builds the authoritative 0-5 map at runtime from UsageError/AuthError/exitCodeForStatus and asserts both README exit-code tables (codes, row counts, and per-language meanings), the AGENTS.md contract line, and every Troubleshooting (exit `N`) reference against it — the last hand-copied contract now has a lock. A repository-name lock derives the canonical slug from package.json#repository and checks every github.com / img.shields.io URL across README.md, README.zh-CN.md, AGENTS.md, and CHANGELOG.md, plus a hard ban on the pre-rename yuque/yuque-cli slug (the drift that actually shipped once). Sibling-project links are explicitly allowlisted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/e2e/schema-registry.ts compiles every mapped operation's response and requestBody schema from spec/yuque-openapi.yaml (Ajv2020 for the 3.1 dialect, local $ref dereferencing, path-template matching including %2F-encoded namespaces). FixtureServer now validates request bodies before dispatch and response bodies before send; a violation 500s the request and rethrows the detailed Ajv error at stop(), failing the test. The gate immediately caught four real fixture hallucinations, now fixed: format 'sheet' → 'lakesheet' (+type Sheet), an invalid 'board' format (→ type Board), and stats member counters that the spec declares as strings. Known spec-vs-live-API mismatches (user groups and the three statistics collections are lists, not single objects) get operation-scoped array corrections consistent with the existing note in src/client/api/stats.ts — no blanket loosening. Boundary: response schemas carry no required/additionalProperties, so the gate catches type/enum/structure errors but not extra hallucinated response fields; tightening via injected additionalProperties:false was evaluated and deliberately deferred (spec gaps + --json passthrough). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
src/client/types.gen.ts is now produced by openapi-typescript from spec/yuque-openapi.yaml (npm run gen:types); the committed output is byte-compared in the merge gate via gen:types:check, so a spec refresh without regeneration fails check. src/client/types.ts shrinks to a 63-line adapter that keeps every public type name, restores the [key: string]: unknown index signatures the --json passthrough contract relies on, and re-types relation fields with the adapted types. The hand-written mirror is gone: 118 spec fields the manual types never declared are now typed, 75 type/optionality mismatches now follow the spec, and exactly one manual extension survives (V2DocDetail.last_editor, documented inline). Known live-API divergences stay runtime-only with scoped casts and comments (stats member user object, legacy sheet/table formats) instead of widening the generated types. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes every cli:null entry in the constraint-pin table: - search --offset <n> — the spec's legacy page-number parameter (wire name offset, range 1-100); kept distinct from the existing --page and documented as a page number, not a record offset - doc get --page / --page-size — data-table content paging on all three doc-detail operations (page >= 1, page_size 1-200); omitted from the wire when not passed doc get's action now follows the AGENTS.md-pinned order (local validation before getContext), so a bad target with a missing token exits 2 instead of 3, matching the documented convention. The stale process.exit claim in search.ts's validation comment — the same misinformation corrected in group.ts/stats.ts during phase 1 — is rewritten to match the empirically verified behavior. Constraint pins, help-surface golden, unit wire assertions, mock-e2e coverage, and both READMEs updated per the AGENTS.md checklist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2543198af9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| export type V2DocVersionDetail = PublicSchemaWith<'V2DocVersionDetail', { user?: V2User }>; | ||
| export type V2TocItem = PublicSchema<'V2TocItem'>; | ||
| export type V2GroupStatistics = PublicSchema<'V2GroupStatistics'>; | ||
| export type V2MemberStatistics = PublicSchema<'V2MemberStatistics'>; |
There was a problem hiding this comment.
Define this with PublicSchemaWith and override user as V2User, as is done for the other relation-bearing types. Live statistics rows contain user: { name: ... }—the command tests model that shape and MEMBER_COLUMNS consumes it—but this alias inherits the spec's incorrect user?: string, forcing the unsafe double cast in the command and giving consumers of listMemberStatistics the wrong type.
AGENTS.md reference: AGENTS.md:L50-L50
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds out the repo's agent-iteration harness so that every class of change gets deterministic programmatic feedback, then closes the CLI/spec parameter gaps the new locks surfaced.
Fixes & hardening (phase 1)
exitOverridebehavior (redundant.exitOverride()removed, pinned by the new flag-validation test)smoke:dist/test:e2enow build first — the stale-dist trap is gone;checkshares one buildno-floating-promiseset al.),--max-warnings 0, prettier over scripts/configs, coverage thresholds enforced incheckDocs harness (phase 2)
tests/docs/agents-md.test.ts(everysrc/**/*.tspath must be mapped; documented npm scripts must exist)Programmatic locks (B-series)
tests/docs/help-surface.test.ts): every leaf command's flags/args pinned; hallucinated flags fail deterministically; README flag reverse-locktests/spec-constraints.test.ts): every enum/min/max/required in mapped operations pinned bidirectionally + parametrized CLI boundary tests (169 cases)tests/docs/exit-code-docs.test.ts): both READMEs + AGENTS.md checked againstsrc/errors.tsat runtime; canonical repo slug pinned to package.json, old slug bannedtests/e2e/schema-registry.ts): ajv validates every fixture response/request body against the OpenAPI schemas — caught four real fixture hallucinationssrc/client/types.gen.ts): openapi-typescript from the spec with a byte-exact drift check incheck;types.tsis now a thin adapter preserving the--jsonpassthrough index signaturesSpec-gap flags (follow-up the locks surfaced)
search --offset(legacy page number, 1–100),doc get --page/--page-size(data-table paging, 1–200) — closing everycli: nullconstraint pinBefore/after on the four hallucination-injection experiments (all were silent green before): hallucinated flag → red, spec constraint tamper → red, stale-dist run → red,
any/floating promise → red.npm run check(unchanged single merge gate, ~8s): 390 unit tests + 38 mock-e2e, all green on every commit in this branch.🤖 Generated with Claude Code