[ST-NNNN] Tag-based test execution filtering by gmedori · Pull Request #3239 · swiftlang/swift-evolution

gmedori · 2026-04-13T04:24:06Z

For convenience, here's a link to the implementation of the feature described in this proposal: swiftlang/swift-testing#1531

grynspan · 2026-04-13T14:16:19Z

proposals/testing/nnnn-tag-based-test-execution-filtering.md

+
+## Future Directions
+
+N/A


Consider whether there are other aspects of tests on which devs may want to filter?

Ah fair point! I actually wrote this a little while ago to address an open github issue and am just putting up the PR now, and I didn't stop to think about future directions. Lemme give it a little thought.

Added some thoughts in 61e2f4b

proposals/testing/nnnn-tag-based-test-execution-filtering.md

grynspan · 2026-04-13T14:18:06Z

proposals/testing/nnnn-tag-based-test-execution-filtering.md

+
+The example above would behave as though the string `tag:uiTest` were passed as a regular expression, omitting the escaping backslash in the final regular expression.
+
+> **Note**: Most shells treat the backslash character `\` as a special character used for escaping. In order for the application to receive it, the argument needs to either be wrapped in quotes like `'tag\:uiTest'`, or the backslash itself needs to be escaped, `tag\\:uiTest`.


Should we just make it a policy that \ escapes the next character? If not, consider adding some discussion in this area to Alternatives Considered.

Or, if that's impractical or undesirable, should we document how to write \: verbatim (i.e. we probably also need to allow escaping \ itself)?

Personally I think that if we want to make that a policy, the policy should exist at SPM level for every command. I'm not sure it's useful to have a policy that only applies to swift testing. Probably skews impractical IMO. Think this is something we want to pursue anyway?

Or, if that's impractical or undesirable, should we document how to write : verbatim (i.e. we probably also need to allow escaping \ itself)?

Do you mean in the usage description of the --skip and --filter options?

grynspan · 2026-04-13T14:20:12Z

proposals/testing/nnnn-tag-based-test-execution-filtering.md

+
+## Integration with Supporting Tools
+
+If a codebase has test functions or suites that contain the string `tag:`, then any filtering/skipping arguments that begin with `tag:` behave differently with this proposal. They are treated as tags rather than regular expressions.


This discussion might belong in Source Compatibility rather than this section because it doesn't really affect e.g. VS Code or Xcode integrations.

Yeah I read through that last night and think I agree.

I moved this discussion to Source Compatibility and added some clearer thoughts around tool integrations in 9005487

allevato · 2026-04-13T14:30:20Z

proposals/testing/nnnn-tag-based-test-execution-filtering.md

+
+### Handling Raw Identifiers
+
+ As of [SE-0451](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0451-escaped-identifiers.md), Swift has raw identifiers which means the following is valid Swift:


You should also consider what needs to happen (if anything different) if the tag itself is a raw identifier (unless anything else in Swift Testing prohibits that, which I don't know off the top of my head):

@Test(.tags(.`some tag with spaces`)) func `my test`() { ... }

I think I would expect to write swift test --skip 'tag:some tag with spaces' here.

Then there's an open question about whether or not developers should specify backticks (which also affect shell processing).

ha, yep that complicates things. I think the most correct answer here is to write it as @allevato put it. Involving backticks opens the command up to different interpretations in different shells, where as single-quoting is relatively universal, even going back to shells like sh and csh.

Added some more discussion around this scenario in 9005487

As a parallel, when using raw identifiers in the -module-alias or -module-abi-name command line flags of the compiler, you don't need to include the backticks: you'd write -module-abi-name 'foo bar', not -module-abi-name '`foo bar`'. So I think it's reasonable to have the same policy here.

In other words, the backticks are required to delimit the identifier in source code (and they also appear in the mangling of the symbol for compatibility reasons), but it's reasonable to drop them elsewhere.

I don't have an opinion either way, but we should be clear about it. (And if the developer does type the backticks, does that make it wrong?)

…the discussion of raw identifiers

Add initial draft for nnnn-tag-based-test-execution-filtering.md

8953883

gmedori mentioned this pull request Apr 13, 2026

Request for the ability to run tests in a Package with a given tag from the command line swiftlang/swift-testing#591

Open

grynspan reviewed Apr 13, 2026

View reviewed changes

proposals/testing/nnnn-tag-based-test-execution-filtering.md Outdated Show resolved Hide resolved

grynspan reviewed Apr 13, 2026

View reviewed changes

proposals/testing/nnnn-tag-based-test-execution-filtering.md Outdated Show resolved Hide resolved

grynspan reviewed Apr 13, 2026

View reviewed changes

allevato reviewed Apr 13, 2026

View reviewed changes

gmedori added 3 commits April 13, 2026 16:01

Add ideas for future directions we could take skipping/filtering

61e2f4b

Fix some markdown formatting

d80d77b

Reorganized some stuff around source compatibility and added more to …

9005487

…the discussion of raw identifiers


		The example above would behave as though the string `tag:uiTest` were passed as a regular expression, omitting the escaping backslash in the final regular expression.

		> Note: Most shells treat the backslash character `\` as a special character used for escaping. In order for the application to receive it, the argument needs to either be wrapped in quotes like `'tag\:uiTest'`, or the backslash itself needs to be escaped, `tag\\:uiTest`.


		## Integration with Supporting Tools

		If a codebase has test functions or suites that contain the string `tag:`, then any filtering/skipping arguments that begin with `tag:` behave differently with this proposal. They are treated as tags rather than regular expressions.


		### Handling Raw Identifiers

		As of [SE-0451](https://github.com/swiftlang/swift-evolution/blob/main/proposals/0451-escaped-identifiers.md), Swift has raw identifiers which means the following is valid Swift:

Conversation

gmedori commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

grynspan Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gmedori Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gmedori commented Apr 13, 2026 •

edited

Loading

grynspan Apr 13, 2026 •

edited

Loading

gmedori Apr 13, 2026 •

edited

Loading