Skip to content

evaluate-relevance file-no-longer-exists false-positives on upstream-plugin paths #220

@tompahoward

Description

@tompahoward

Description

wr-itil:review-problems Step 4.6 relevance-close invokes wr-itil-evaluate-relevance (canonical body packages/itil/scripts/evaluate-relevance.sh) against each aged ticket. The file-no-longer-exists shape (ADR-079 Phase 1) extracts paths from the ticket body and verifies each via git ls-files --error-unmatch against the current repo. For tickets that track upstream-plugin bugs, the cited paths live in the upstream plugin source tree at ~/.claude/plugins/marketplaces/<marketplace>/packages/<plugin>/..., NOT in the consumer repo. The evaluator correctly observes "these paths are not in git ls-files" but misclassifies them as file-no-longer-exists rather than "file is upstream and not expected to be local", producing CLOSE-CANDIDATE / CLOSE-CANDIDATE-WITH-CAVEAT false-positives.

Symptoms

  • wr-itil-evaluate-relevance emits CLOSE-CANDIDATE on tickets whose body cites paths beginning with packages/<plugin>/, .changeset/, etc. - paths that conventionally live in an upstream plugin's source tree.
  • The cites: field names each plugin path verbatim as "absent from this repo" when they are authoritatively expected to be absent (the plugin lives upstream).
  • The surface-batch-confirm flow catches these, but the false-positive adds disposition cost on every retro / review pass.

Workaround

The surface-batch-confirm flow already catches the false-positives: the user reviews each CLOSE-CANDIDATE-WITH-CAVEAT and dispositions it "keep" for plugin-tracking tickets. Non-blocking but adds friction every review cycle.

Affected plugin or component

@windyroad/itil - packages/itil/scripts/evaluate-relevance.sh (the file-no-longer-exists shape).

Frequency

Every /wr-itil:review-problems invocation, every retro, every relevance-close pass, for any consumer repo whose backlog tracks @windyroad/* plugin bugs.

Versions

  • Local plugin: @windyroad/itil@0.44.0
  • Upstream package: @windyroad/itil@0.44.0
  • Claude Code CLI: 2.1.150 (Claude Code)
  • Node: v22.17.1
  • OS: Darwin 25.3.0 x86_64

Evidence

Proposed fix (option 1, lightest): when a cited path matches packages/<plugin>/ AND the same path resolves under ~/.claude/plugins/marketplaces/*/packages/<plugin>/, suppress the file-no-longer-exists cite for that path - check the standard plugin install location after the local-repo check. Zero configuration; infers from the installed plugin set. Reproduced 2026-06-02 against 005-external-comms-marker... and 006-capture-problem-deferred-refresh... (both track active upstream bugs; paths live in the plugin tree, not the consumer repo).

Cross-reference

Reported from https://github.com/mountain-pass/addressr-mcp docs/problems/known-error/010-relevance-evaluator-false-positive-on-upstream-plugin-paths.md

Tracked locally as P010 in the downstream project's docs/problems/ directory.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions