migrate-xunit-to-mstest: add learnings from dotnet/sdk migration by Evangelink · Pull Request #756 · dotnet/skills

Evangelink · 2026-06-11T20:41:38Z

Folds learnings from a recent xUnit → MSTest migration in dotnet/sdk (5 stacked PRs: dotnet/sdk#54722, #54723, #54724, #54725, #54726) back into this skill.

Changes

1. `MSTest.Sdk` implicit global using (new)

MSTest.Sdk adds Microsoft.VisualStudio.TestTools.UnitTesting as an implicit global using. The skill previously didn't say so, which led to:

<Using Include="Microsoft.VisualStudio.TestTools.UnitTesting" /> being added to every migrated csproj (redundant noise; flagged in review).
using Microsoft.VisualStudio.TestTools.UnitTesting; being added to every test file (also redundant).

Added a callout in SKILL.md Step 2 (Option B) and Step 3 and a section in cheatsheet §9 (Option B) with Do not bullets. Option A (the MSTest metapackage) still needs the per-file using -- noted explicitly.

2. `Assert.IsExactInstanceOfType<T>` (closes #755)

xUnit's Assert.IsType<T> is exact type. The current cheatsheet recommended Assert.IsInstanceOfType<T> (assignable-from) plus a follow-up Assert.AreEqual(typeof(T), x.GetType()) workaround. MSTest 4.1+ has Assert.IsExactInstanceOfType<T> (microsoft/testfx#4633) -- single call, exact match, returns T. Same goes for Assert.IsNotExactInstanceOfType<T>.

This was a real regression in dotnet/sdk#54722 (39 sites silently weakened from exact to assignable, caught by a reviewer; re-flipped in a follow-up commit). Updated SKILL.md inline mapping table and cheatsheet §3.3 with the modern API; kept the pre-4.1 fallback as a note for legacy projects.

3. `Assert.AreSequenceEqual` (MSTest 4.3+)

The cheatsheet recommended CollectionAssert.AreEqual(expected.ToList(), actual.ToList()) for xUnit's element-wise Assert.Equal on IEnumerable<T>. MSTest 4.3+ added Assert.AreSequenceEqual(expected, actual) which:

accepts IEnumerable<T> directly (no .ToList()),
avoids CollectionAssert,
avoids the MSTEST0065 trap where plain Assert.AreEqual silently does a reference compare on collections.

Updated cheatsheet §3.6 to recommend AreSequenceEqual (with pre-4.3 fallback noted).

4. `AwesomeAssertions` namespace note

Section 10 already said "keep -- assertion libraries are framework-agnostic" for AwesomeAssertions. Added a one-line clarification that it ships in the FluentAssertions namespace for API compat, so the swap is no-source-change. (This came up during PR review where a reviewer mistook <Using Include="FluentAssertions" /> for a leftover.)

Out of scope

Removing the StringAssert.* fallback from cheatsheet §3.5 -- the Assert.Contains/StartsWith/EndsWith flat-namespace API has been around since MSTest 3.8 and is the recommended target for current migrations, but the fallback is still useful for very old projects. Left alone.

Validation

Both files still under skill-validator's advisory limits (SKILL.md: 412 lines; cheatsheet: 307 lines).
All learnings verified end-to-end against MSTest 4.3 in dotnet/sdk (282 tests across 5 projects, all green).

cc /cc @nohwnd @Youssef1313

Refines the skill based on a real migration of 5 test projects in dotnet/sdk: - MSTest.Sdk implicit global using: do not add `<Using Include='Microsoft.VisualStudio.TestTools.UnitTesting' />` or per-file `using` in MSTest.Sdk projects (it's already in scope). Note Option A (`MSTest` metapackage) still needs the per-file using. - `Assert.IsExactInstanceOfType<T>` (MSTest 4.1+) is the proper single-call equivalent of xUnit's exact-type `Assert.IsType<T>`; previous guidance silently degraded it to assignable semantics (closes dotnet#755). - `Assert.AreSequenceEqual` (MSTest 4.3+) is the modern element-wise equivalent of xUnit's `Assert.Equal` on `IEnumerable<T>`, avoiding the `CollectionAssert` + `.ToList()` dance and the MSTEST0065 trap on plain `AreEqual`. - Clarify `AwesomeAssertions` ships in the `FluentAssertions` namespace, so it's a no-source-change swap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Folds learnings discovered while migrating dotnet/sdk to MSTest.Sdk: - MSTest.Sdk implicit global using note - Assert.IsExactInstanceOfType<T> (MSTest 4.1+) for xUnit Assert.IsType<T> - Assert.AreSequenceEqual (MSTest 4.3+) over CollectionAssert.AreEqual - AwesomeAssertions namespace clarification Keeps the sdk-local copy in sync with the upstream PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Updates the migrate-xunit-to-mstest skill documentation to incorporate recent learnings from dotnet/sdk’s xUnit → MSTest migration, with a focus on preserving assertion semantics and reducing migration noise when using MSTest.Sdk.

Changes:

Document MSTest.Sdk’s implicit global using for Microsoft.VisualStudio.TestTools.UnitTesting and warn against redundant per-project/per-file additions.
Update type-check guidance to use Assert.IsExactInstanceOfType<T> / Assert.IsNotExactInstanceOfType<T> (MSTest 4.1+) to preserve xUnit’s exact-type semantics.
Update collection equality guidance to prefer Assert.AreSequenceEqual (MSTest 4.3+) with a pre-4.3 fallback.

Show a summary per file

File	Description
plugins/dotnet-test/skills/migrate-xunit-to-mstest/SKILL.md	Adds guidance about `MSTest.Sdk` implicit global using and updates assertion mapping guidance.
plugins/dotnet-test/skills/migrate-xunit-to-mstest/references/mapping-cheatsheet.md	Refreshes the cheatsheet with newer MSTest APIs and adds clarifying notes for common migration pitfalls.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 2/2 changed files
Comments generated: 2

… equivalence Address dotnet/sdk#54727 review comment: the Step 6 summary table conflated xUnit's exact-type Assert.IsType<T> and assignable Assert.IsAssignableFrom<T> into a single row, which implies both map interchangeably even though their semantics differ. Split into three rows -- IsType<T> -> IsExactInstanceOfType<T>, IsNotType<T> -> IsNotExactInstanceOfType<T>, IsAssignableFrom<T> -> IsInstanceOfType<T> -- and explicitly call out the silent-weakening trap. The cheatsheet already maps these correctly; this aligns the SKILL.md summary with the cheatsheet so the at-a-glance table doesn't suggest a wrong mapping. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

… equivalence Addresses review comment from copilot-pull-request-reviewer: the Step 6 summary table conflated xUnit's exact-type Assert.IsType<T> and assignable Assert.IsAssignableFrom<T> into a single row, which implies both map interchangeably even though their semantics differ. Split into three rows -- IsType<T> -> IsExactInstanceOfType<T>, IsNotType<T> -> IsNotExactInstanceOfType<T>, IsAssignableFrom<T> -> IsInstanceOfType<T> -- and explicitly call out the silent-weakening trap. The cheatsheet already mapped these correctly; this aligns the SKILL.md summary so the at-a-glance table doesn't suggest a wrong mapping. Mirrored in dotnet/skills#756. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…on B Address review feedback on dotnet#756: Step 3 said to skip the per-file 'using Microsoft.VisualStudio.TestTools.UnitTesting;' on Option B (MSTest.Sdk provides it as an implicit global using), but Step 4 unconditionally instructed replacing xUnit usings with that MSTest using -- so an agent following Step 4 literally would re-add the redundant using on Option B projects. Also drop the dangling 'Step 4 rewriter' phrasing in Step 3 (there is no rewriter in Step 4, just a list of mechanical rewrites the agent applies). Step 4 now explicitly says to skip the MSTest using on Option B and only remove the 'using Xunit;' lines, matching Step 2's note and Step 3's branch instructions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Evangelink · 2026-06-11T21:08:16Z

/evaluate

Copilot

Copilot's findings

Files reviewed: 2/2 changed files
Comments generated: 0 new

github-actions · 2026-06-11T21:27:26Z

Skill Validation Results

Skill	Scenario	Quality	Skills Loaded	Overfit	Verdict
migrate-xunit-to-mstest	Migrate basic xUnit v2 project to MSTest v4 preserving VSTest	4.3/5 → 4.0/5 🔴	✅ migrate-xunit-to-mstest; tools: skill	✅ 0.10	❌ [1]
migrate-xunit-to-mstest	Migrate basic xUnit v3 project to MSTest v4 preserving VSTest	5.0/5 → 3.0/5 🔴	✅ migrate-xunit-to-mstest; tools: skill, create / ✅ migrate-xunit-to-mstest; tools: skill	✅ 0.10	❌
migrate-xunit-to-mstest	Stop when target framework is unsupported by MSTest v4	1.3/5 → 2.0/5 🟢	✅ migrate-xunit-to-mstest; tools: skill / ✅ migrate-xunit-to-mstest; tools: skill, bash	✅ 0.10	✅ [2]
migrate-xunit-to-mstest	Convert IClassFixture to ClassInitialize	3.0/5 → 3.0/5	✅ migrate-xunit-to-mstest; tools: skill / ✅ migrate-xunit-to-mstest; tools: skill, create	✅ 0.10	❌ [3]
migrate-xunit-to-mstest	Handle ICollectionFixture explicitly (do not silently widen scope)	2.7/5 → 3.3/5 🟢	✅ migrate-xunit-to-mstest; tools: skill / ✅ migrate-xunit-to-mstest; tools: skill, bash, read_bash	✅ 0.10	✅
migrate-xunit-to-mstest	Convert ITestOutputHelper to TestContext	5.0/5 → 5.0/5	✅ migrate-xunit-to-mstest; tools: skill, create / ✅ migrate-xunit-to-mstest; tools: skill	✅ 0.10	❌ [4]
migrate-xunit-to-mstest	Map exception assertions correctly (xUnit Throws -> MSTest ThrowsExactly)	4.7/5 → 4.3/5 🔴	✅ migrate-xunit-to-mstest; tools: skill, create / ✅ migrate-xunit-to-mstest; tools: skill	✅ 0.10	❌ [5]
migrate-xunit-to-mstest	Convert MemberData and TheoryData to DynamicData	3.3/5 → 3.3/5	✅ migrate-xunit-to-mstest; tools: skill	✅ 0.10	❌ [6]
migrate-xunit-to-mstest	Convert Skip, Trait, and Timeout	3.3/5 → 3.3/5	✅ migrate-xunit-to-mstest; tools: skill, create / ✅ migrate-xunit-to-mstest; tools: skill, read_bash	✅ 0.10	✅ [7]
migrate-xunit-to-mstest	Preserve xUnit parallelization default with [assembly: Parallelize]	3.7/5 → 3.3/5 🔴	✅ migrate-xunit-to-mstest; tools: skill	✅ 0.10	✅ [8]
migrate-xunit-to-mstest	Convert xUnit v3 TestContext.Current.CancellationToken	1.0/5 → 1.0/5	✅ migrate-xunit-to-mstest; tools: skill / ✅ migrate-xunit-to-mstest; tools: skill, edit, bash, read_bash, create	✅ 0.10	❌ [9]
migrate-xunit-to-mstest	Recognize project already on MSTest	5.0/5 → 5.0/5	✅ migrate-xunit-to-mstest; tools: skill	✅ 0.10	❌ [10]

[1] ⚠️ High run-to-run variance (CV=237%) — consider re-running with --runs 5
[2] ⚠️ High run-to-run variance (CV=52%) — consider re-running with --runs 5
[3] ⚠️ High run-to-run variance (CV=110%) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -1.6% due to: tokens (55743 → 74878)
[4] (Plugin) Quality unchanged but weighted score is -6.2% due to: tokens (69294 → 187321), time (36.4s → 48.7s)
[5] ⚠️ High run-to-run variance (CV=94%) — consider re-running with --runs 5
[6] ⚠️ High run-to-run variance (CV=2909%) — consider re-running with --runs 5. (Isolated) Quality unchanged but weighted score is -5.6% due to: tokens (55229 → 110965)
[7] ⚠️ High run-to-run variance (CV=584%) — consider re-running with --runs 5
[8] ⚠️ High run-to-run variance (CV=264%) — consider re-running with --runs 5. (Isolated) Quality dropped but weighted score is +12.6% due to: efficiency metrics
[9] (Isolated) Quality unchanged but weighted score is -10.8% due to: judgment, tokens (17038 → 21343)
[10] ⚠️ High run-to-run variance (CV=62%) — consider re-running with --runs 5. (Plugin) Quality unchanged but weighted score is -4.1% due to: tokens (25626 → 79026), time (10.0s → 18.0s), tool calls (3 → 4)

Model: claude-opus-4.6 | Judge: claude-opus-4.6

🔍 Full Results - additional metrics and failure investigation steps

To investigate failures, paste this to your AI coding agent:

For PR 756 in dotnet/skills, download eval artifacts with gh run download 27377541255 --repo dotnet/skills --pattern "skill-validator-results-*" --dir ./eval-results, then fetch https://raw.githubusercontent.com/dotnet/skills/d8c00eb0d7767e214c41498720c6d7d2e97b9630/eng/skill-validator/src/docs/InvestigatingResults.md and follow it to analyze the results.json files. Diagnose each failure, suggest fixes to the eval.yaml and skill content, and tell me what to fix first.

▶ Sessions Visualisation -- interactive replay of all evaluation sessions
📊 Session Analytics (preview) -- aggregated metrics across evaluation sessions

github-actions · 2026-06-11T22:00:39Z

✅ Evaluation passed for d8c00eb. cc @dotnet/dotnet-testing — please review.

Copilot AI review requested due to automatic review settings June 11, 2026 20:41

Evangelink mentioned this pull request Jun 11, 2026

migrate-xunit-to-mstest: recommend Assert.IsExactInstanceOfType for xUnit Assert.IsType #755

Closed

Copilot started reviewing on behalf of Evangelink June 11, 2026 20:42 View session

Evangelink mentioned this pull request Jun 11, 2026

Add migrate-xunit-to-mstest skill from dotnet/skills dotnet/sdk#54727

Open

Copilot AI reviewed Jun 11, 2026

View reviewed changes

Comment thread plugins/dotnet-test/skills/migrate-xunit-to-mstest/SKILL.md

Comment thread plugins/dotnet-test/skills/migrate-xunit-to-mstest/SKILL.md Outdated

Copilot AI review requested due to automatic review settings June 11, 2026 21:07

Copilot started reviewing on behalf of Evangelink June 11, 2026 21:08 View session

github-actions Bot added pr-state/ready-for-eval PR is mergeable and awaiting evaluation evaluate-now Trigger evaluation.yml for current PR head (transient) labels Jun 11, 2026

Copilot AI reviewed Jun 11, 2026

View reviewed changes

Evangelink enabled auto-merge (squash) June 11, 2026 21:22

github-actions Bot added waiting-on-review PR state label and removed pr-state/ready-for-eval PR is mergeable and awaiting evaluation labels Jun 11, 2026

JanKrivanek approved these changes Jun 12, 2026

View reviewed changes

Evangelink merged commit 9be44ad into dotnet:main Jun 12, 2026
35 of 37 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

migrate-xunit-to-mstest: add learnings from dotnet/sdk migration#756

migrate-xunit-to-mstest: add learnings from dotnet/sdk migration#756
Evangelink merged 3 commits into
dotnet:mainfrom
Evangelink:evangelink/migrate-xunit-mstest-learnings

Evangelink commented Jun 11, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Evangelink commented Jun 11, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Jun 11, 2026

Uh oh!

github-actions Bot commented Jun 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Evangelink commented Jun 11, 2026

Changes

1. MSTest.Sdk implicit global using (new)

2. Assert.IsExactInstanceOfType<T> (closes #755)

3. Assert.AreSequenceEqual (MSTest 4.3+)

4. AwesomeAssertions namespace note

Out of scope

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Evangelink commented Jun 11, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

github-actions Bot commented Jun 11, 2026

Skill Validation Results

Uh oh!

github-actions Bot commented Jun 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

1. `MSTest.Sdk` implicit global using (new)

2. `Assert.IsExactInstanceOfType<T>` (closes #755)

3. `Assert.AreSequenceEqual` (MSTest 4.3+)

4. `AwesomeAssertions` namespace note