What needs to be fixed?
Parent context: RFC #156 / #159 — bundle tests (skills/**/test_skill.py) now run in CI via pytest skills/. After merge, novelty_extractor failed because bundle tests called real fastembed, which tried to download from HuggingFace (429 rate limits). The hotfix mocks embeddings in bundle tests; docs should make the rule explicit so the next skill does not repeat this.
Problem: Docs already say bundle tests should be "offline" and "mock externals", but they do not spell out that model downloads (HuggingFace, Ollama, first-run embedding weights) count as network and must be mocked in bundle tests. TESTING.md also notes [all] includes fastembed, which contributors may read as "run real fastembed in CI".
Deliverables:
-
docs/TESTING.md — In the Skill bundle test section, add bullets:
- Bundle tests run in CI on every PR — no live HTTP, no API keys, no downloading models.
- Mock HTTP, LLM clients, and embedding/model loaders (
fastembed, etc.).
[all] satisfies imports; real inference belongs in maintainer tests (tests/skills/) or local runs, not bundle CI gates.
-
CONTRIBUTING.md — Under test_skill.py (bundle test), add one line: mock network and first-run model downloads.
-
templates/python_skill/README.md — Step 7: extend "mock externals" to include embedding/model loaders.
-
docs/contributing/ai_native_workflow.md — Skill PR checklist: add item that bundle tests mock all network and model downloads (CI runs pytest skills/).
-
Optional: docs/skills/novelty_extractor.md — One sentence under testing: bundle tests mock embeddings; real fastembed is for local use and examples.
Out of scope:
Acceptance criteria:
- A new contributor reading only TESTING + CONTRIBUTING understands bundle tests must not download models in CI.
- No stale wording implying bundle tests only run locally.
- Wording stays concise; no issue/RFC links required in doc body (repo preference).
Affected Page
docs/TESTING.md, CONTRIBUTING.md, templates/python_skill/README.md, docs/contributing/ai_native_workflow.md (optional: docs/skills/novelty_extractor.md)
What needs to be fixed?
Parent context: RFC #156 / #159 — bundle tests (
skills/**/test_skill.py) now run in CI viapytest skills/. After merge,novelty_extractorfailed because bundle tests called realfastembed, which tried to download from HuggingFace (429 rate limits). The hotfix mocks embeddings in bundle tests; docs should make the rule explicit so the next skill does not repeat this.Problem: Docs already say bundle tests should be "offline" and "mock externals", but they do not spell out that model downloads (HuggingFace, Ollama, first-run embedding weights) count as network and must be mocked in bundle tests.
TESTING.mdalso notes[all]includesfastembed, which contributors may read as "run real fastembed in CI".Deliverables:
docs/TESTING.md— In the Skill bundle test section, add bullets:fastembed, etc.).[all]satisfies imports; real inference belongs in maintainer tests (tests/skills/) or local runs, not bundle CI gates.CONTRIBUTING.md— Undertest_skill.py(bundle test), add one line: mock network and first-run model downloads.templates/python_skill/README.md— Step 7: extend "mock externals" to include embedding/model loaders.docs/contributing/ai_native_workflow.md— Skill PR checklist: add item that bundle tests mock all network and model downloads (CI runspytest skills/).Optional:
docs/skills/novelty_extractor.md— One sentence under testing: bundle tests mock embeddings; realfastembedis for local use and examples.Out of scope:
skill.pyor bundle test implementations (hotfix handlesnovelty_extractor)Acceptance criteria:
Affected Page
docs/TESTING.md, CONTRIBUTING.md, templates/python_skill/README.md, docs/contributing/ai_native_workflow.md (optional: docs/skills/novelty_extractor.md)