Skip to content

[Feat]: PyPI wheel smoke test — load every registry skill after pip install #182

Description

@rosspeili

Feature Description

Clone-repo tests (pytest tests/, pytest skills/) do not prove that a built PyPI wheel ships all registry bundles correctly. Packaging regressions (__init__.py, MANIFEST.in, package-data) can slip through until a user runs pip install skillware in a clean project.

Add a CI job (or step after build) that:

  1. Builds the wheel/sdist from the PR branch (python -m build or pip wheel .).
  2. Creates a fresh venv with no editable install.
  3. pip install the built artifact with [all] (or minimal + per-skill extras as needed).
  4. For each skills/**/manifest.yaml, runs SkillLoader.load_skill("<category>/<name>") and asserts bundle keys (manifest, instructions, module).
  5. Optionally runs one offline assertion per skill (manifest name matches folder id).

Parent context: RFC #156 testing architecture. Complements bundle tests in editable mode; does not replace pytest skills/.

Rationale

Skillware's core promise is pip install skillware → load skills by registry ID. Wheel-only breaks are rare but painful (missing __init__.py, assets not in wheel, wrong package-data). A smoke test catches them before release and gives contributors confidence that packaging changes are safe.

This is higher leverage than adding more maintainer tests under tests/skills/ because it validates the shipping path agents and PyPI users actually use.

Implementation Idea

  • Add .github/workflows/wheel-smoke.yml (or extend ci.yml with a wheel-smoke job).
  • Script: scripts/wheel_smoke_test.py invoked from CI.
  • Use dummy env vars already set in CI (ETHERSCAN_API_KEY, ANTHROPIC_API_KEY) where load triggers warnings only.
  • Document in docs/TESTING.md under a short "Packaging smoke test" bullet.
  • CHANGELOG [Unreleased] when merged.

Metadata

Metadata

Assignees

No one assigned

    Labels

    core frameworkChanges to loader, env, or base classes.enhancementNew feature or requesthelp wantedExtra attention is needed
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions