feat: fix AI false-positive breaking verdicts; add acknowledgedChanges#57
Merged
Conversation
Three fixes for #56, where the reviewer confirmed two backwards-compatible changes as breaking and returned "confirmed" rather than a downgrade, so --ai-apply-downgrades had nothing to act on. - Prompt rules: adding an optional property (including one widened into a Pick/Omit) is non-breaking, and no verdict may stay conditional on a fact already resolvable from context. - Usage-site context: the focused verdict prompt now lists each changed type's referrers so the model can tell a read-only output type from a consumer-constructed input. Referrers are gathered across the package's sibling subpath surfaces (the type and the function returning it usually live in different rollups), threaded via AiPackageContext.siblingCurrentApiJsonPaths and matched by canonicalReference; an unresolved reference falls back to breaking. - acknowledgedChanges config: unconditionally downgrade a verified-safe breaking change by name or glob and tag it in the report. Both AI fixes only ever enable a downgrade, still gated behind --ai-apply-downgrades, so the default path cannot clear a real break.
commit: |
Contributor
API Changes Report
Summary
@clerk/break-checkCurrent version: 0.0.1 🟢 Additions (2)Added:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #56, where the AI reviewer confirmed two backwards-compatible changes as breaking (clerk/javascript#8700) and returned "confirmed" rather than a downgrade, so
--ai-apply-downgradeshad nothing to act on.The optional-
Pickcase was a pure reasoning gap (the referenced definition was already in the model's context), so it's fixed with prompt rules. The output-type case needed structure, not prompting: the focused verdict context only ever showed a type's forward references, never its callers, so the model couldn't tell a read-only response type from one consumers construct. It now ships a "Usage sites" block of each changed type's referrers. The load-bearing part is that those referrers are gathered across the package's sibling subpath surfaces, because the analyzer runs per-subpath and the changed type (@clerk/shared/types) and the function that returns it (@clerk/shared/react) live in different rollups.detector.tsthreads the sibling current.api.jsonpaths in; matching is bycanonicalReference, and anything unresolved (including cross-package usage) falls back to breaking.Also adds
acknowledgedChanges, a config allowlist to green a verified-safe break by name or glob. Unlike an AI downgrade it's unconditional and not gated behind a flag.Both AI fixes only ever enable a downgrade, which still stays behind
--ai-apply-downgrades, so the default path still can't clear a real break. The directionality rule is a pragmatic heuristic (a consumer could construct any exported type), which is the judgment worth a close look in review.The two live regression tests in
ai-verdict.test.mjsreproduce the exact false positives but are gated on a real API key; the nightly ai-smoke workflow runs them. Everything else is deterministic and runs inpnpm test.