Skip to content

fix: replace stale flat command references and guard llms.txt freshness#607

Merged
djm81 merged 3 commits into
devfrom
bugfix/stale-flat-refs-llms-freshness
Jun 12, 2026
Merged

fix: replace stale flat command references and guard llms.txt freshness#607
djm81 merged 3 commits into
devfrom
bugfix/stale-flat-refs-llms-freshness

Conversation

@djm81

@djm81 djm81 commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • replace remaining flat command references missed by fix: align flat command diagnostics #606: runtime error suggestions (specfact analyze/specfact reprospecfact code analyze contracts/specfact code repro), prompt validator command map (specfact enforce sdd/specfact sync bridge → grouped forms), and export-change-to-github docstrings
  • plan verb references stay on the legacy surface deliberately — their deprecation/rehome is tracked in [Change] Deprecate former plan backlog-edit verbs and re-home project migration specfact-cli-modules#317
  • add tests/unit/docs/test_llms_overview_freshness.py: re-runs the command overview generator in --check mode so a stale llms.txt or generated command reference fails the test suite even when the path-scoped pre-commit gate is bypassed (merge commits, --no-verify, bot commits); skips when the companion modules checkout is unavailable
  • ignore local-only .agents/skills/ so repo lint gates skip them
  • bump to 0.47.7 with changelog entry

A companion freshness test lands in specfact-cli-modules (its CI skips overview validation on dev-to-main promote runs — the bigger gap).

Validation

  • new freshness test passes locally against the sibling modules checkout
  • hatch run lint, check-version-sources, and all pre-commit blocks passed on commit

🤖 Generated with Claude Code

- Update runtime error suggestions from removed flat commands to canonical
  grouped commands (specfact code analyze contracts, specfact code repro,
  specfact project sync bridge, specfact govern enforce sdd).
- Update validate_prompts CLI command map for grouped commands; plan verbs
  stay legacy until nold-ai/specfact-cli-modules#317 rehomes them.
- Fix export-change-to-github docstrings to specfact project sync bridge.
- Add tests/unit/docs/test_llms_overview_freshness.py re-running the command
  overview generator in --check mode so stale llms.txt or generated command
  references fail the suite even when the path-scoped pre-commit gate is
  bypassed (merge commits, --no-verify, bot commits).
- Ignore local-only agent skills so repo lint gates skip them.
- Bump version artifacts to 0.47.7 with changelog entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@strix-security

Copy link
Copy Markdown

Strix is installed on this repository, but we could not run this PR security review because this workspace does not have an active plan. If you'd like to continue receiving code reviews, you can add a payment method or manage billing here.

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 18b487a7-a0d2-475a-9379-8d8979664b62

📥 Commits

Reviewing files that changed from the base of the PR and between edd6eb2 and 79359ce.

📒 Files selected for processing (1)
  • tests/unit/docs/test_llms_overview_freshness.py
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • nold-ai/specfact-cli-modules (manual)
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/unit/docs/test_llms_overview_freshness.py
📜 Recent review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
  • GitHub Check: License Compliance Gate
  • GitHub Check: Security Audit (pip-audit)
  • GitHub Check: CLI Command Validation
  • GitHub Check: Type Checking (basedpyright)
  • GitHub Check: Contract-First CI
  • GitHub Check: Tests (Python 3.12)
  • GitHub Check: Linting (ruff, pylint, safe-write guard)
  • GitHub Check: Compatibility (Python 3.11)
  • GitHub Check: Docs Review
  • GitHub Check: Contract Validation
🧰 Additional context used
🔀 Multi-repo context nold-ai/specfact-cli-modules

nold-ai/specfact-cli-modules

  • llms.txt present at repo root (104 lines). [::nold-ai/specfact-cli-modules::llms.txt]
  • Command-overview generator exists: scripts/generate-command-overview.py (references LLMS_PATH and prints stale-artifact message). [::nold-ai/specfact-cli-modules::scripts/generate-command-overview.py:23,238]
  • CI / pre-commit integrate the generator and llms.txt:
    • pyproject hatch tasks: generate-command-overview / check-command-overview. [::nold-ai/specfact-cli-modules::pyproject.toml:58-59]
    • .github workflow includes scripts/generate-command-overview.py and llms.txt. [::nold-ai/specfact-cli-modules::.github/workflows/docs-review.yml:17,19,37,39]
    • scripts/pre-commit-quality-checks.sh regenerates/stages llms.txt and generated command artifacts and runs check-command-overview. [::nold-ai/specfact-cli-modules::scripts/pre-commit-quality-checks.sh:81,169-181,187-192]
  • Tests and specs assert or mention legacy flat commands (will be impacted by wording changes):
    • tests/unit/test_check_prompt_commands_script.py contains expectations for "specfact repro --repo ." and "specfact sync bridge --help". [::nold-ai/specfact-cli-modules::tests/unit/test_check_prompt_commands_script.py:74,90,137,146,201,214]
    • tests/e2e/specfact_project/test_help_smoke.py asserts "specfact sync bridge" absence. [::nold-ai/specfact-cli-modules::tests/e2e/specfact_project/test_help_smoke.py:29]
    • openspec specs and change docs contain many occurrences of "specfact sync bridge" and other flat forms. [::nold-ai/specfact-cli-modules::openspec/**]
  • Repo already contains many references to the new grouped canonical commands (so PR replacements align with module artifacts):
    • llms.txt and generated command docs list specfact code repro entries. [::nold-ai/specfact-cli-modules::llms.txt:54-55]
    • packages/specfact-spec and packages/specfact-codebase reference specfact code analyze contracts and specfact code repro. [::nold-ai/specfact-cli-modules::packages/specfact-spec/src/specfact_spec/generate/commands.py:541,966,1510; ::nold-ai/specfact-cli-modules::packages/specfact-codebase/src/specfact_codebase/analyze/commands.py:75-79]
  • New freshness test in specfact-cli will call the generator in the modules repo (matches scripts/generate-command-overview.py behavior); paired-worktree layouts are considered by the new test. See test file in modules pointing to generator. [::nold-ai/specfact-cli-modules::tests/unit/docs/test_llms_overview_freshness.py:1,20,22,34,47-48]
  • Docs and resources rely on generated artifacts (commands.generated.json/md, docs references) which the PR's --check test aims to validate; these files exist in repo (docs/reference/commands.generated.*). [::nold-ai/specfact-cli-modules::docs/reference/commands.generated.json; ::nold-ai/specfact-cli-modules::docs/reference/commands.generated.md]

Implication: The specfact-cli PR’s changes (suggestion strings, prompt-command mapping, and the new freshness test) are directly relevant to this modules repo because it is the source/consumer of llms.txt and generated command references; tests and docs here contain both legacy flat commands and canonical grouped commands that the PR updates aim to reconcile. [::nold-ai/specfact-cli-modules::]


📝 Walkthrough

User-Visible Behavior & CLI Surface

The PR updates CLI command references across documentation and error messaging to use grouped canonical commands (from the "flat" deprecated forms):

  • specfact analyze / specfact analyze --bundlespecfact code analyze contracts --bundle
  • specfact reprospecfact code repro
  • specfact enforce sddspecfact govern enforce sdd
  • specfact sync bridgespecfact project sync bridge

These changes affect:

  • Runtime suggestion strings in src/specfact_cli/utils/suggestions.py (displayed to users when errors occur or when providing improvement guidance)
  • Export-to-GitHub workflow documentation in scripts/export-change-to-github.py
  • Prompt validator command map in tools/validate_prompts.py (used for CLI alignment testing)

Plan verb references remain intentionally on the legacy surface until tracked rehome (nold-ai/specfact-cli-modules#317).

Contract & API Impact

No changes to public function signatures, Pydantic models, or module boundaries.

The prompt validator's CLI_COMMANDS lookup table in tools/validate_prompts.py has been updated to validate against the new grouped command forms. This affects what command strings are expected within prompt templates (specfact.05-enforce, specfact.06-sync, specfact.07-contracts, specfact.validate).

Testing & Quality Gates

New freshness test suite (tests/unit/docs/test_llms_overview_freshness.py):

  • test_generated_command_artifacts_exist(): Asserts required generated files exist (llms.txt, docs/reference/commands.generated.*)
  • test_llms_and_command_overview_are_current(): Runs scripts/generate-command-overview.py in --check mode; skips gracefully when companion modules checkout is unavailable (supports both direct sibling and paired-worktree layouts via specfact-cli-worktrees), ensuring test suite fails even if path-scoped pre-commit gates are bypassed

The test includes helper functions _paired_worktree_modules_repo() and _modules_repo_available() to detect companion module checkouts flexibly.

Version & Changelog

  • Version bump: 0.47.6 → 0.47.7 across pyproject.toml, setup.py, src/__init__.py, src/specfact_cli/__init__.py
  • CHANGELOG.md: Added entry documenting stale flat command reference fixes and llms.txt freshness test
  • Additional: .agents/skills/ added to lint ignore scope (local-only directory)

Validation: New freshness test passed locally against sibling modules checkout; hatch run lint, check-version-sources, and pre-commit blocks passed on commit.

Walkthrough

This release bumps the package version to 0.47.7 and refactors CLI command references across suggestion helpers, documentation, and validation tools to use a reorganized command hierarchy. A new freshness test validates that generated command documentation remains in sync.

Changes

Version 0.47.7 Release

Layer / File(s) Summary
Version Bump (0.47.6 → 0.47.7)
pyproject.toml, setup.py, src/__init__.py, src/specfact_cli/__init__.py
Package version incremented uniformly across all version declaration points.
CLI Command Reference Updates
scripts/export-change-to-github.py, src/specfact_cli/utils/suggestions.py, tools/validate_prompts.py
Command strings updated to reflect refactored hierarchy: specfact code analyze contracts (was specfact analyze), specfact project sync bridge (was specfact sync bridge), specfact code repro (was specfact repro), and specfact govern enforce sdd (was specfact enforce sdd). Updates applied consistently across docstrings, suggestion text, and prompt validation expectations.
Generated Artifact Freshness Test
tests/unit/docs/test_llms_overview_freshness.py
New test module with helpers to detect specfact-cli-modules availability (including paired worktree detection) and two test cases: one asserting that llms.txt and docs/reference/commands.generated.* files exist; another re-running the command-overview generator in --check mode with timeout to catch stale artifacts, skipping gracefully when modules repo is unavailable.
Release Changelog Entry
CHANGELOG.md
Documents version 0.47.7 fixes for stale command reference suggestions and introduces the new llms.txt freshness test.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested labels

bug

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ❓ Inconclusive Description provides clear summary of changes, validation approach, and notes on cross-repo impact; however, it lacks required template sections (Type of Change checklist, Contract-First Testing sections, Test Execution, and Quality Gates Status). Consider completing the PR template sections (Type of Change, Testing Evidence, Test Execution checklist) to provide readers with explicit confirmation of what was tested and validated before merge.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed Title uses conventional commit 'fix:' prefix and accurately summarizes the main changes: replacing stale command references and adding llms.txt freshness guardrails.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch bugfix/stale-flat-refs-llms-freshness

Comment @coderabbitai help to get the list of available commands and usage tips.

@djm81
djm81 changed the base branch from main to dev June 12, 2026 21:54
@github-actions

Copy link
Copy Markdown
Contributor

SpecFact CLI Validation Report

All validations passed!
Duration: 19.21s
Checks: 4 total (1 passed) (3 skipped)

@github-actions

Copy link
Copy Markdown
Contributor

SpecFact CLI Validation Report

All validations passed!
Duration: 19.13s
Checks: 4 total (1 passed) (3 skipped)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: edd6eb2e21

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/unit/docs/test_llms_overview_freshness.py
@djm81 djm81 self-assigned this Jun 12, 2026
@djm81 djm81 added documentation Improvements or additions to documentation enhancement New feature or request labels Jun 12, 2026
@djm81 djm81 moved this from Todo to In Progress in SpecFact CLI Jun 12, 2026
@djm81 djm81 added the architecture Architecture and design changes label Jun 12, 2026
@djm81 djm81 linked an issue Jun 12, 2026 that may be closed by this pull request
5 tasks
The generator resolves the companion modules checkout via SPECFACT_MODULES_REPO,
a direct sibling, or the paired specfact-cli-worktrees layout; the freshness
test's skip helper only checked the first two, so it skipped in paired-worktree
checkouts where it could run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@djm81 djm81 removed the architecture Architecture and design changes label Jun 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

SpecFact CLI Validation Report

All validations passed!
Duration: 20.51s
Checks: 4 total (1 passed) (3 skipped)

@djm81
djm81 merged commit 0e3c4d3 into dev Jun 12, 2026
20 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in SpecFact CLI Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[Change] Remove flat alias module diagnostics from root CLI errors

1 participant