Feature Description
Registry skills frequently land with correct bundles but stale or missing index rows in contributor-facing docs. Today this is caught only in manual review (e.g. missing agent_loops.md row for defi/evm_tx_handler, incomplete CONTRIBUTING category examples).
Add tests/test_registry_docs.py (framework layer, runs in pytest tests/) with automated parity checks:
- Catalog parity — count of
skills/**/manifest.yaml equals skill rows in docs/skills/README.md (parse table rows with backtick skill IDs; exclude section headers).
- Catalog page exists — each manifest has
docs/skills/<skill_name>.md (derive skill_name from path).
- Examples index parity — every
examples/*.py (except *_common.py if grandfathered) appears in examples/README.md Runnable Scripts table; flag orphaned index rows pointing to missing files.
- Agent loops matrix (soft check) — every registry skill ID appears at least once in
docs/usage/agent_loops.md table (may be (catalog page) or script name).
On first add, fix any failures or document a short grandfather list with TODO + issue links. Relates to #104 (provider sections on catalog pages) but does not duplicate it.
Rationale
Low effort, high impact: turns recurring maintainer review comments into a failing CI signal. Prevents "skill merged but invisible in docs" drift as the registry grows (#148 wellness, future skills).
Complements #160 (bundle test presence) and #178 (example script correctness) without overlapping — this issue is index completeness, not tool-name dispatch or provider snippet content.
Implementation Idea
Feature Description
Registry skills frequently land with correct bundles but stale or missing index rows in contributor-facing docs. Today this is caught only in manual review (e.g. missing
agent_loops.mdrow fordefi/evm_tx_handler, incomplete CONTRIBUTING category examples).Add
tests/test_registry_docs.py(framework layer, runs inpytest tests/) with automated parity checks:skills/**/manifest.yamlequals skill rows indocs/skills/README.md(parse table rows with backtick skill IDs; exclude section headers).docs/skills/<skill_name>.md(derive skill_name from path).examples/*.py(except*_common.pyif grandfathered) appears inexamples/README.mdRunnable Scripts table; flag orphaned index rows pointing to missing files.docs/usage/agent_loops.mdtable (may be(catalog page)or script name).On first add, fix any failures or document a short grandfather list with TODO + issue links. Relates to #104 (provider sections on catalog pages) but does not duplicate it.
Rationale
Low effort, high impact: turns recurring maintainer review comments into a failing CI signal. Prevents "skill merged but invisible in docs" drift as the registry grows (#148 wellness, future skills).
Complements #160 (bundle test presence) and #178 (example script correctness) without overlapping — this issue is index completeness, not tool-name dispatch or provider snippet content.
Implementation Idea
tests/test_registry_docs.pydocs/TESTING.md§ Framework tests.examples/*.pyfor tool names ([Docs]: Audit runnable examples for manifest-derived tool name dispatch #178).