Use deterministic pre-step for re-quarantine detection to bypass DIFC filtering#66707
Merged
Merged
Conversation
The MCP search_pull_requests tool applies a DIFC integrity filter that silently removes PRs authored by external contributors (e.g., author_association=CONTRIBUTOR). This caused PR dotnet#49307 (re-quarantine of SetsTlsHandshakeFeatureForHttps by captainsafia) to be invisible, leading to the test being incorrectly unquarantined repeatedly. Switch to curl/python3 with the GitHub Search API for the re-quarantine check only. The bash tools are not subject to DIFC filtering. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
the
needs-area-label
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the test quarantine agent instructions so re-quarantine detection uses direct GitHub REST API calls instead of the MCP search tool, avoiding integrity-filtered search results.
Changes:
- Adds explicit guidance to use
curl/python3for re-quarantine PR searches. - Provides GitHub Search API examples for label/title-based re-quarantine lookup.
- Adds deduplication and PR file-fetching guidance for matching changed test files.
Comments suppressed due to low confidence (1)
.github/workflows/test-quarantine.md:227
- This request only reads the first page of changed files for each PR. The pull request files endpoint is paginated, so a re-quarantine PR touching more than 100 files could hide the relevant test file on a later page and be missed. Please add pagination here as well before deciding that a PR did not re-quarantine the test.
curl -s "https://api.github.com/repos/dotnet/aspnetcore/pulls/{PR_NUMBER}/files?per_page=100"
The agent needs to check the diff (patch field) for added lines containing [QuarantinedTest], not just whether the file was touched. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Use GITHUB_TOKEN auth header to avoid rate limits - Follow pagination via Link header for all searches - If any API call fails, skip ALL unquarantine actions (fail-safe) - Fix unclosed bracket: [QuarantinedTest -> [QuarantinedTest] - Switch from curl to python3/urllib for consistency Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Read GITHUB_TOKEN from os.environ in python3, never pass it on the command line where it would appear in agent logs. Removed Authorization header from the example snippets since they're just URL references now. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move the re-quarantine PR search into a pre-activation step that runs with the workflow's GITHUB_TOKEN before the agent starts. This: 1. Bypasses DIFC integrity filtering (the root cause of the bug) 2. Eliminates any risk of token leakage in agent logs 3. Writes results to /tmp/requarantine-prs.json for the agent to read The agent now just reads a JSON file instead of making API calls. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…n, fail-closed - Store added_lines (not just filenames) in JSON for method/class/assembly matching - Match [QuarantinedTest (without closing bracket) to catch parameterized forms - Handle truncated patches by failing closed (treat whole file as re-quarantined) - Add fail-closed instruction: if JSON is missing/unparseable, skip all unquarantines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
wtgodbe
changed the title
github-actions
Bot
added
area-infrastructure
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The quarantine workflow's MCP
search_pull_requeststool applies a DIFC integrity filter that silently removes PRs authored by external contributors (author_association=CONTRIBUTOR). This caused re-quarantine PRs like #49307 to be invisible to the agent, leading to incorrect unquarantine decisions.Changes
on.steps:step in the gh-aw frontmatter that runs a Python script usingGITHUB_TOKENto fetch all merged re-quarantine PRs (by label or title) and their diffs via the GitHub REST API[QuarantinedTest(not just filenames), enabling the agent to match at method/class/assembly level/tmp/requarantine-prs.jsoninstead of using MCP search toolsWhy not MCP search?
The MCP
search_pull_requeststool silently filters results based on the author's association with the repository. PRs from users withCONTRIBUTORassociation (external contributors who have had PRs merged) are excluded. Since the quarantine process predates the workflow, many quarantine issues and PRs were created by humans who may be external contributors. The deterministic pre-step bypasses this filter entirely.