migrate-xunit-to-mstest: forbid CancellationToken.None replacement explicitly

[Evangelink]

Why

The TestContext.Current.CancellationToken guidance in migrate-xunit-to-mstest already forbids replacing it with a freshly-constructed CancellationTokenSource, but real-world agent migrations have been observed substituting CancellationToken.None instead, which silently drops the test host's cancellation linkage in the same way.

What

Update the REQUIRED for CancellationToken callout to call out both shortcuts explicitly:

Do NOT replace TestContext.Current.CancellationToken with CancellationToken.None or a freshly-constructed CancellationTokenSource -- both lose the test-host's cancellation linkage and change behavior under timeouts.

Source

Observed during the dotnet/sdk xUnit→MSTest migration: see e.g. dotnet/sdk#54725 (BrowserRefresh.Tests) and dotnet/sdk#54731 (HotReload.Client.Tests), where the migration agent rewrote TestContext.Current.CancellationToken as CancellationToken.None in every WriteAsync/FlushAsync call. Restoring the original behavior required adding public TestContext TestContext { get; set; } to the test class and using TestContext.CancellationToken.

[Copilot]

Pull request overview

This PR updates the migrate-xunit-to-mstest skill documentation to more explicitly forbid a common incorrect migration pattern where TestContext.Current.CancellationToken is replaced with CancellationToken.None, which drops the test-host cancellation linkage (similar to using a new CancellationTokenSource).

Changes:

• Expand the “REQUIRED for CancellationToken” callout to explicitly forbid CancellationToken.None in addition to freshly-constructed CancellationTokenSource.

Show a summary per file

File	Description
plugins/dotnet-test/skills/migrate-xunit-to-mstest/SKILL.md	Strengthens migration guidance to preserve cancellation behavior under timeouts by forbidding CancellationToken.None substitutions.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 1/1 changed files
• Comments generated: 1

[YuliiaKovalova]

/evaluate

[github-actions]

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
migrate-xunit-to-mstest	Migrate basic xUnit v2 project to MSTest v4 preserving VSTest	4.0/5 → 3.7/5 🔴	✅ migrate-xunit-to-mstest; tools: skill, create / ✅ migrate-xunit-to-mstest; tools: skill, read_bash	✅ 0.08	❌ [1]
migrate-xunit-to-mstest	Migrate basic xUnit v3 project to MSTest v4 preserving VSTest	5.0/5 → 3.3/5 🔴	✅ migrate-xunit-to-mstest; tools: skill	✅ 0.08	❌
migrate-xunit-to-mstest	Stop when target framework is unsupported by MSTest v4	1.7/5 → 2.0/5 🟢	✅ migrate-xunit-to-mstest; tools: skill, bash / ✅ migrate-xunit-to-mstest; tools: skill, read_bash, bash	✅ 0.08	✅ [2]
migrate-xunit-to-mstest	Convert IClassFixture to ClassInitialize	4.0/5 → 3.3/5 🔴	✅ migrate-xunit-to-mstest; tools: skill / ✅ migrate-xunit-to-mstest; tools: skill, create, read_bash	✅ 0.08	❌ [3]
migrate-xunit-to-mstest	Handle ICollectionFixture explicitly (do not silently widen scope)	2.3/5 → 3.3/5 🟢	✅ migrate-xunit-to-mstest; tools: skill, create / ✅ migrate-xunit-to-mstest; tools: skill, read_bash, bash	✅ 0.08	✅ [4]
migrate-xunit-to-mstest	Convert ITestOutputHelper to TestContext	5.0/5 → 5.0/5	✅ migrate-xunit-to-mstest; tools: skill, create / ✅ migrate-xunit-to-mstest; tools: skill	✅ 0.08	❌ [5]
migrate-xunit-to-mstest	Map exception assertions correctly (xUnit Throws -> MSTest ThrowsExactly)	4.3/5 → 4.3/5	✅ migrate-xunit-to-mstest; tools: skill, create / ✅ migrate-xunit-to-mstest; tools: skill	✅ 0.08	❌ [6]
migrate-xunit-to-mstest	Convert MemberData and TheoryData to DynamicData	3.3/5 → 3.7/5 🟢	✅ migrate-xunit-to-mstest; tools: skill	✅ 0.08	❌ [7]
migrate-xunit-to-mstest	Convert Skip, Trait, and Timeout	3.7/5 → 3.0/5 🔴	✅ migrate-xunit-to-mstest; tools: skill, create	✅ 0.08	❌ [8]
migrate-xunit-to-mstest	Preserve xUnit parallelization default with [assembly: Parallelize]	3.0/5 → 2.7/5 🔴	✅ migrate-xunit-to-mstest; tools: skill	✅ 0.08	✅ [9]
migrate-xunit-to-mstest	Convert xUnit v3 TestContext.Current.CancellationToken	1.0/5 → 1.0/5	⚠️ NOT ACTIVATED / ✅ migrate-xunit-to-mstest; tools: report_intent, skill, view, bash, edit, read_bash	✅ 0.08	✅ [10]
migrate-xunit-to-mstest	Recognize project already on MSTest	5.0/5 → 5.0/5	✅ migrate-xunit-to-mstest; tools: skill / ✅ migrate-xunit-to-mstest; tools: skill, glob	✅ 0.08	❌ [11]

[1] ⚠️ High run-to-run variance (CV=112%) — consider re-running with --runs 5
[2] ⚠️ High run-to-run variance (CV=101%) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=107%) — consider re-running with --runs 5
[4] ⚠️ High run-to-run variance (CV=362%) — consider re-running with --runs 5
[5] ⚠️ High run-to-run variance (CV=64%) — consider re-running with --runs 5. (Plugin) Quality unchanged but weighted score is -5.5% due to: tokens (69134 → 197440), time (33.1s → 52.4s), tool calls (8 → 10)
[6] ⚠️ High run-to-run variance (CV=3537%) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -15.9% due to: judgment, tokens (63187 → 111320)
[7] (Isolated) Quality improved but weighted score is -3.2% due to: tokens (55221 → 92811)
[8] ⚠️ High run-to-run variance (CV=68%) — consider re-running with --runs 5
[9] ⚠️ High run-to-run variance (CV=182%) — consider re-running with --runs 5. (Isolated) Quality dropped but weighted score is +15.2% due to: efficiency metrics
[10] ⚠️ High run-to-run variance (CV=103%) — consider re-running with --runs 5
[11] (Plugin) Quality unchanged but weighted score is -8.3% due to: tokens (25623 → 89472), time (9.7s → 20.3s), tool calls (3 → 4)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

To investigate failures, paste this to your AI coding agent:

For PR 758 in dotnet/skills, download eval artifacts with gh run download 27418330953 --repo dotnet/skills --pattern "skill-validator-results-*" --dir ./eval-results, then fetch https://raw.githubusercontent.com/dotnet/skills/3491c1de6df389adbafcc486bfb705b93a3888e0/eng/skill-validator/src/docs/InvestigatingResults.md and follow it to analyze the results.json files. Diagnose each failure, suggest fixes to the eval.yaml and skill content, and tell me what to fix first.

▶ Sessions Visualisation -- interactive replay of all evaluation sessions
📊 Session Analytics (preview) -- aggregated metrics across evaluation sessions