Skip to content

[Feat]: CI doc-drift guards — skill catalog and examples index parity #183

Description

@rosspeili

Feature Description

Registry skills frequently land with correct bundles but stale or missing index rows in contributor-facing docs. Today this is caught only in manual review (e.g. missing agent_loops.md row for defi/evm_tx_handler, incomplete CONTRIBUTING category examples).

Add tests/test_registry_docs.py (framework layer, runs in pytest tests/) with automated parity checks:

  1. Catalog parity — count of skills/**/manifest.yaml equals skill rows in docs/skills/README.md (parse table rows with backtick skill IDs; exclude section headers).
  2. Catalog page exists — each manifest has docs/skills/<skill_name>.md (derive skill_name from path).
  3. Examples index parity — every examples/*.py (except *_common.py if grandfathered) appears in examples/README.md Runnable Scripts table; flag orphaned index rows pointing to missing files.
  4. Agent loops matrix (soft check) — every registry skill ID appears at least once in docs/usage/agent_loops.md table (may be (catalog page) or script name).

On first add, fix any failures or document a short grandfather list with TODO + issue links. Relates to #104 (provider sections on catalog pages) but does not duplicate it.

Rationale

Low effort, high impact: turns recurring maintainer review comments into a failing CI signal. Prevents "skill merged but invisible in docs" drift as the registry grows (#148 wellness, future skills).

Complements #160 (bundle test presence) and #178 (example script correctness) without overlapping — this issue is index completeness, not tool-name dispatch or provider snippet content.

Implementation Idea

Metadata

Metadata

Assignees

Labels

No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions