skill: add /triage-e2e-test#14896
Open
midleman wants to merge 10 commits into
Open
Conversation
|
E2E Tests 🚀 Note No feature tags detected. If this PR needs feature coverage, add the tag above and retrigger the workflow. |
midleman
marked this pull request as draft
July 15, 2026 20:54
Manual-only triage skill (disable-model-invocation). Name makes the already-failing-in-CI scope explicit and points new-test authoring to author-e2e-tests.
midleman
force-pushed
the
mi/fluttering-cookie
branch
from
July 15, 2026 21:10
5231f6b to
8db6db2
Compare
midleman
changed the title
…2e-test skill - e2e-query-history.js and e2e-failure-analyzer/SKILL.md fall back to reading E2E_INSIGHTS_API_KEY from .env.e2e when the env var isn't set - triage-e2e-test/SKILL.md: full describe-chain test keys, JSON array --test-keys, table-based failure mode summary, dominance-based triage priority, collaborative root-cause sleuthing, reproduce-before-fixing guidance, and notes on 403 report-url fallback, raw log fallback when the mined excerpt doesn't explain the mechanism, and verifying prior-art fixes via repro before trusting them
…ft rubric category Learnings from a real triage: an ambiguous 69/31 failure split turned out to be one bug with two error messages, and the smoking gun was an aria-live status announcement contradicting the test's own error, not the DOM markup.
selectMetadataOption() clicked the metadata button directly, which silently no-ops when the button's click handler reads a not-yet-updated active console session from React context right after a focus change. Reuse the existing openMetadataDialog() retry helper, already used elsewhere in this file for the same race.
…yzer scripts Guard --test-id in SKILL.md's evidence-pull snippet so it falls back to --title when report_url has no testId fragment; fix truthy-vs-null checks for --occurrences-per-pattern/--max-patterns so "0" is forwarded correctly; trim .env.e2e keys before comparison; match full hierarchical test titles in matchesFilter instead of leaf-only.
…dataOption" This reverts commit d327bda.
List triage-e2e-test as a third consumer that must stay in sync. Fix the evidence-reading list contradicting its own "read error-context first" instruction, define infrastructure-issue vs test-environment-issue, make timeout a last-resort category instead of an undefined default, align category naming in the locator-drift section, and scope "check the triggering commit" to single-run analysis with a bridge for consumers reasoning over many historical occurrences instead.
midleman
marked this pull request as ready for review
July 16, 2026 20:38
midleman
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a manual-only
/triage-e2e-testskill: starts from a known failing/flaky test, surfaces its recent distinct failure modes from history, pulls evidence per mode, and traces each to its actual root cause with a proposed fix.e2e-failure-analyzerscripts:.claude/skills/e2e-failure-analyzer/scripts/e2e-query-history.js--occurrences-per-pattern/--max-patterns.claude/skills/e2e-failure-analyzer/scripts/e2e-process-s3.js**.env.e2efallback forE2E_INSIGHTS_API_KEYQA Notes
Depends on posit-dev/e2e-test-insights#186 for per-pattern evidence.