From 58b60c9ca413d7169349d76d9e08c855957ffb66 Mon Sep 17 00:00:00 2001 From: wigodbe Date: Tue, 12 May 2026 08:10:49 -0700 Subject: [PATCH 1/2] Fix re-quarantine check: don't append test name to search query The agent was adding the test class name to the GitHub search query for re-quarantine PRs (e.g., 'label:re-quarantine TlsHandshakeFeatureTests'), but PR #49307's title uses the method name ('SetsTlsHandshakeFeatureForHttps'), causing 0 results. The instructions now explicitly say NOT to append the test name to the search query, and instead filter by checking changed files. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/test-quarantine.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.github/workflows/test-quarantine.md b/.github/workflows/test-quarantine.md index 70eeea8e2f20..2b231dd656ca 100644 --- a/.github/workflows/test-quarantine.md +++ b/.github/workflows/test-quarantine.md @@ -205,7 +205,10 @@ A test is a candidate for unquarantining if ALL of the following are true: git log --format="%H %ai" -1 -G 'QuarantinedTest.*{ISSUE_NUMBER}' -- {FILE_PATH} ``` If the commit date is less than 60 days ago, skip this test — it was recently quarantined and needs more time to establish reliability. -- The test has **never been re-quarantined**. A test is considered re-quarantined if there exists any merged PR in the repository that either has "Re-quarantine" (case-insensitive) in the title, or has the `re-quarantine` label, and that PR modified the same test file to add a `[QuarantinedTest]` attribute for this test. To check this, search for merged PRs with the `re-quarantine` label using `search_issues` (query: `repo:dotnet/aspnetcore is:pr is:merged label:re-quarantine`), and also search for merged PRs with "Re-quarantine" in the title (query: `repo:dotnet/aspnetcore is:pr is:merged "Re-quarantine" in:title`). For each matching PR, check its changed files — if any touch the same file and test, this test must be permanently excluded from automated unquarantining. Only a human may unquarantine such a test. +- The test has **never been re-quarantined**. A test is considered re-quarantined if there exists any merged PR in the repository that either has "Re-quarantine" (case-insensitive) in the title, or has the `re-quarantine` label, and that PR modified the same test file to add a `[QuarantinedTest]` attribute for this test. To check this: + 1. Search for merged PRs with the `re-quarantine` label: `repo:dotnet/aspnetcore is:pr is:merged label:re-quarantine` (do **not** append the test name to this query — PR titles often use method names, class names, or abbreviations that won't match a text search). + 2. Search for merged PRs with "Re-quarantine" in the title: `repo:dotnet/aspnetcore is:pr is:merged "Re-quarantine" in:title` (again, do **not** append the test name). + 3. For each matching PR from either search, check its changed files using `pull_request_read` (method `get_files`). If any changed file matches the test's source file path, this test must be permanently excluded from automated unquarantining. Only a human may unquarantine such a test. For IIS tests compiled into multiple assemblies (Common.LongTests, Common.FunctionalTests), the same test method appears with different namespace prefixes (e.g., `FunctionalTests.StartupTests.X`, `IISExpress.FunctionalTests.StartupTests.X`, `NewHandler.FunctionalTests.StartupTests.X`, `NewShim.FunctionalTests.StartupTests.X`). ALL variants must have 100% pass rates. Variants with 0 pass / 0 fail (all "other" outcomes) represent tests skipped by `[ConditionalFact]` and should be excluded from the pass-rate check — they are neither passing nor failing. From 47bb74912c8b4948a54f73a328c1cb1dbb9f2e48 Mon Sep 17 00:00:00 2001 From: wigodbe Date: Tue, 12 May 2026 08:16:48 -0700 Subject: [PATCH 2/2] Broaden re-quarantine check to cover class and assembly level MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The check should exclude a test if any re-quarantine PR added a [QuarantinedTest] attribute to the test method, its containing class, or its assembly — not just if the PR touched the same file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/test-quarantine.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test-quarantine.md b/.github/workflows/test-quarantine.md index 2b231dd656ca..b03b73d741d5 100644 --- a/.github/workflows/test-quarantine.md +++ b/.github/workflows/test-quarantine.md @@ -205,10 +205,10 @@ A test is a candidate for unquarantining if ALL of the following are true: git log --format="%H %ai" -1 -G 'QuarantinedTest.*{ISSUE_NUMBER}' -- {FILE_PATH} ``` If the commit date is less than 60 days ago, skip this test — it was recently quarantined and needs more time to establish reliability. -- The test has **never been re-quarantined**. A test is considered re-quarantined if there exists any merged PR in the repository that either has "Re-quarantine" (case-insensitive) in the title, or has the `re-quarantine` label, and that PR modified the same test file to add a `[QuarantinedTest]` attribute for this test. To check this: +- The test has **never been re-quarantined**. A test is considered re-quarantined if there exists any merged PR in the repository that either has "Re-quarantine" (case-insensitive) in the title, or has the `re-quarantine` label, and that PR added a `[QuarantinedTest]` attribute to the same test method, test class, or test assembly. To check this: 1. Search for merged PRs with the `re-quarantine` label: `repo:dotnet/aspnetcore is:pr is:merged label:re-quarantine` (do **not** append the test name to this query — PR titles often use method names, class names, or abbreviations that won't match a text search). 2. Search for merged PRs with "Re-quarantine" in the title: `repo:dotnet/aspnetcore is:pr is:merged "Re-quarantine" in:title` (again, do **not** append the test name). - 3. For each matching PR from either search, check its changed files using `pull_request_read` (method `get_files`). If any changed file matches the test's source file path, this test must be permanently excluded from automated unquarantining. Only a human may unquarantine such a test. + 3. For each matching PR from either search, check its changed files using `pull_request_read` (method `get_files`). If any changed file adds a `[QuarantinedTest]` attribute to the test method, the test's containing class, or the test's assembly, this test must be permanently excluded from automated unquarantining. Only a human may unquarantine such a test. For IIS tests compiled into multiple assemblies (Common.LongTests, Common.FunctionalTests), the same test method appears with different namespace prefixes (e.g., `FunctionalTests.StartupTests.X`, `IISExpress.FunctionalTests.StartupTests.X`, `NewHandler.FunctionalTests.StartupTests.X`, `NewShim.FunctionalTests.StartupTests.X`). ALL variants must have 100% pass rates. Variants with 0 pass / 0 fail (all "other" outcomes) represent tests skipped by `[ConditionalFact]` and should be excluded from the pass-rate check — they are neither passing nor failing.