Skip to content

ci(agent-skills): seed update tests without live setup fetches #632

Description

@devantler

🤖 Generated by the Daily AI Engineer

Objective

Remove redundant live gh skill install calls that only prepare the update-agent-skills CI tests, reducing the self-inflicted GitHub API burst tracked by #514 without reducing the distinct setup or update behavior covered by CI.

Evidence and impact

The current agent-skills jobs plan 10 live install calls per CI run. Three are fixture preparation: the Ubuntu/macOS test-update-agent-skills-noop legs and test-update-agent-skills-dry-run each install the same pinned github/awesome-copilot/git-commit skill before exercising the update action. In failing run 29030150199, every API-bearing agent-skills leg failed during the same rate-limit burst.

These preparation calls add load but do not test a separate setup contract. Consumers need the shared actions repository's required CI to remain reliable during dependency-update bursts, while retaining real cross-platform coverage of both composite actions.

Hypothesis and success signal

If the update tests start from a deterministic checked-in pinned-skill provenance fixture, CI can cut planned live install calls from 10 to 7 while preserving every distinct setup/update behavior. The delivery PR's real workflow run is the success signal: setup remains green on Ubuntu and macOS, update remains green on both platforms plus dry-run, and the update jobs emit no setup/install step.

Acceptance criteria

  • Add a deterministic pinned-skill fixture that represents a real installed skill without embedding credentials or mutable generated state.
  • Seed both test-update-agent-skills-noop matrix legs and test-update-agent-skills-dry-run from that fixture instead of invoking ./setup-agent-skills.
  • Keep the real setup-action coverage unchanged: Ubuntu/macOS unpinned installs, pinned install, multi-skill, and multi-agent behavior.
  • Keep the real update-action coverage unchanged: Ubuntu/macOS no-op behavior and unpin dry-run, including output and before/after hash assertions.
  • Add a failing-first structural guard proving update tests cannot silently reintroduce setup-agent-skills fixture preparation.
  • Validate Actionlint, YAML lint, Zizmor, the focused agent-skills tests, and the repository's required CI.
  • Record the observed install-call reduction and keep the separate Zizmor advisories work on parent ci: harden CI against api.github.com secondary rate-limits (zizmor advisories + agent-skills blob 403s) #514.

Dependencies and scope

Part of #514. Active PR #626 changes only the auto-merge workflow and its focused test, so this slice is file-disjoint. No action input, consumer behavior, retry default, or security gate changes.

Rough size

Small.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    ✅ Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions