Skip to content

Add support for Swift Testing in SwiftSyntaxMacrosTestsSupport#3192

Open
0xTim wants to merge 12 commits intoswiftlang:mainfrom
0xTim:support-swift-testing
Open

Add support for Swift Testing in SwiftSyntaxMacrosTestsSupport#3192
0xTim wants to merge 12 commits intoswiftlang:mainfrom
0xTim:support-swift-testing

Conversation

@0xTim
Copy link
Copy Markdown
Member

@0xTim 0xTim commented Nov 21, 2025

  • Explanation:
    As described in Adopt Swift-Testing in test utils such as SwiftSyntaxMacrosTestSupport #2720 SwiftSyntaxMacrosTestsSupport does not work with Swift Testing, which is an issue given Swift Testing is the way to test projects and XCTest is deprecated. If produces false positives (where tests pass even if they shouldn't) which is a major issue, especially as there are no warnings.

This PR adds support for Swift Testing so that tests fail correctly. This does not introduce an issue with a circular dependency on Swift Testing. There is a circular dependency at the package level, but this is allowed due to swiftlang/swift-package-manager#7530. There is no circular dependency between targets.

buildConfiguration: buildConfiguration,
failureHandler: {
#if canImport(Testing)
if Test.current != nil {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not the correct test to determine which library is in use because code can run in a detached task. See swiftlang/swift-testing#475

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the correct way? Should we split it out into a expect function instead?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no correct way at this time, which is why that issue is still open. Jerry's work should allow us to just call #expect() here and have it work under all configurations.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pitch looks like it will solve the issue, but still require work in the library to migrate over to Swift Testing APIs. What I propose is that we land this now, as it solves a problem that exists for users today (and potentially provide a release in the next monthly Linux release/Swift patch release) and then fix forward when the proposal lands. Given it's still in the pitch phase it likely won't be landed until 6.4 and waiting 10 months for a solution seems like a bad idea.

Regarding the Test.current issue - from my understanding this works in all instances apart from those running in a detached task. For this specific case, I can't see a scenario when a user would be using the assertMacroExpansion from a detached task so we can fix this for the majority of the users and those attempting to use it from a detached task will see no change in behaviour.

Copy link
Copy Markdown
Contributor

@grynspan grynspan Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stmontgomery Your take? You okay with this presumably being nonfunctional with the package build?

For this specific case, I can't see a scenario when a user would be using the assertMacroExpansion from a detached task so we can fix this for the majority of the users and those attempting to use it from a detached task will see no change in behaviour.

Let's at least document it as unsupported in the symbol's Markup?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can agree here that SwiftSyntax can provide Testing support by depending on the toolchain's Testing module can we unblock the PR on this matter?

I'd be morally okay with saying "the features in the swift-syntax repository are dependent on the built-in copy of Swift Testing even if you include a package dependency" however this will break builds on non-Apple platforms with flat linker namespaces due to duplicate symbols at link time.

I think there is still the larger question on how this is currently implemented. The if Test.current != nil check in this method is certainly less than ideal. Instead I would propose that we add an entirely new method called expectMacroExpansion that is based on Testing unconditionally. So existing users of assertMacroExpansion can continue to use it with XCTest and Testing users can start adopting the new one. How does that sound to everyone?

Once @jerryjrchen's work on the interop feature lands, it will be possible to implement this in a way that depends on neither XCTest nor Swift Testing. It may be a better idea to just wait until that work is done and revisit the problem at that point.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be morally okay with saying "the features in the swift-syntax repository are dependent on the built-in copy of Swift Testing even if you include a package dependency" however this will break builds on non-Apple platforms with flat linker namespaces due to duplicate symbols at link time.

Only if there is both Testing from the toolchain and from the package right? Which I thought we agreed is only really valid in development environments.

Copy link
Copy Markdown
Contributor

@grynspan grynspan Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only if there is both Testing from the toolchain and from the package right? Which I thought we agreed is only really valid in development environments.

All testing environments are development environments.

If swift-syntax explicitly links the copy of Swift Testing in the toolchain, that will break developers who want to test their macros and have a package dependency on Swift Testing.

In addition, if they are using the package copy of Swift Testing and you are using the toolchain's copy, your calls to e.g. #expect() won't be routed to the infrastructure the developer's test target links against, so a failure will be invisible.

Jerry's work should give us an escape hatch for this problem, so we should wait until it lands and then make the necessary changes here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern with waiting is that it could be a year until that's shipped, meanwhile users have no indications their tests are passing incorrectly, whereas we could fix it today for the majority of use cases

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate the concern, but there are technical blockers here. We must not cause build failures for teams using Swift Testing as a package.

Comment thread Sources/SwiftSyntaxMacrosTestSupport/Assertions.swift Outdated
stmontgomery added a commit to swiftlang/swift-testing that referenced this pull request Jan 22, 2026
…o use a built-in copy (#1452)

This adds a new documentation article within the top-level
`Documentation/` directory describing the various ways Swift Testing is
distributed and offers the specific recommendation that most users
prefer using a built-in copy instead of a package dependency.

View the [rendered
document](https://github.com/stmontgomery/swift-testing/blob/deployment-locations-doc/Documentation/Distributions.md)
for more details.

> Note: In this PR the new article is only being placed in the
`Documentation/` directory. I chose to place it there, initially, and
not within the project's official DocC documentation catalog, but that
could potentially happen in the future.

### Motivation:

We have been casually giving users the recommendation to prefer built-in
copies of the `Testing` module for a while in settings like the Swift
forums, bug reports, and informal chats. But thus far, we have not
collected and formally documented the reasons why using a package copy
can encounter issues. This discussion recently came up during a
[discussion](swiftlang/swift-syntax#3192) about
whether to adopt Swift Testing in a test-helper library in SwiftSyntax.

My hope is that adding this document will help clarify expectations
around the level of support we offer for the two distribution styles for
Swift Testing.

### Checklist:

- [x] Code and documentation should follow the style of the [Style
Guide](https://github.com/apple/swift-testing/blob/main/Documentation/StyleGuide.md).
- [x] If public symbols are renamed or modified, DocC references should
be updated.

---------

Co-authored-by: Joseph Heck <j_heck@apple.com>
@0xTim 0xTim requested a review from hborla as a code owner February 2, 2026 19:35
@0xTim
Copy link
Copy Markdown
Member Author

0xTim commented Feb 2, 2026

Ok all, I've introduced a separate function to be used from Swift testing and added some documentation to the assert functions. Let me know what you think!

cc @FranzBusch

@grynspan
Copy link
Copy Markdown
Contributor

grynspan commented Feb 2, 2026

I think this is still ultimately a moot point with @jerryjrchen's upcoming work?

@FranzBusch
Copy link
Copy Markdown
Member

I think this is still ultimately a moot point with @jerryjrchen's upcoming work?

I thought we identified in the other thread that this is not the case. While the interoperability will allow the existing XCTest based method to work. It won't work with the strict mode which seems limiting. I think the new documentation in the swift-testing makes it pretty clear that the primary use-case of swift-testing as a package is for local development.

@0xTim
Copy link
Copy Markdown
Member Author

0xTim commented Feb 2, 2026

To add some more context as well, I was talking to a couple of maintainers at FOSDEM who have both hit this issue with false positives

I think this is still ultimately a moot point with @jerryjrchen's upcoming work?

This work is still ongoing correct? Because it won't land until 6.4 at the earliest I'd assume

@grynspan
Copy link
Copy Markdown
Contributor

grynspan commented Feb 2, 2026

Right but that doesn't mean we should ship new API that immediately becomes obsolete with the next Swift release.

@grynspan
Copy link
Copy Markdown
Contributor

grynspan commented Feb 2, 2026

While the interoperability will allow the existing XCTest based method to work. It won't work with the strict mode which seems limiting.

The end result of Jerry's work is, as I understand it, that the existing API in this library can adopt #expect() in place of XCTAssertTrue() (etc. etc.) and it'll work in both libraries without problems.

I think the new documentation in the swift-testing makes it pretty clear that the primary use-case of swift-testing as a package is for local development.

I'm not sure I understand how that relates to my comment, but maybe you're referring to something earlier in the conversation I missed?

@jerryjrchen
Copy link
Copy Markdown

While the interoperability will allow the existing XCTest based method to work. It won't work with the strict mode which seems limiting.

We can resolve the strict mode restriction against using XCTest API in a Swift Testing test by adopting #expect in place of XCTest API (I can see XCTAssert* and XCTFail from a quick search). Interop also ensures the conversion to #expect still works as expected if the helpers are being called from an XCTest test case.

@stmontgomery
Copy link
Copy Markdown
Contributor

Hi folks, I want to suggest a path forward here which I hope will unblock progress on this PR and allow it to proceed in the short term (if accepted by the repo's code owners, of course!) while laying out a plan for how it will align with our anticipated and fast-approaching interoperability work.

  1. Keep just one "assert" function, with the existing name assertMacroExpansion()
    • …instead of introducing a separate function expectMacroExpansion() and having one for XCTest, one for Swift Testing.
    • This is to avoid bifurcation between the XCTest and Swift Testing worlds.
  2. Modify assertMacroExpansion() so that it calls both Issue.record() (Swift Testing) and XCTFail() (XCTest), one after the other.
    • For right now, this is sufficient to ensure failures are surfaced regardless of which testing library is active.
    • Note: to achieve full reporting fidelity, the function will need to gain fileID: and column: parameters with default values and plumb those through to the "generic" funnel point so Swift Testing can reference them.
  3. (Important) Communicate the potential effects of this change to SwiftSyntax users
    • This could happen via a detailed release note in the tag this change lands in. See the 602.0.0 notes for example of the format of the last release.
    • This note should do several things:
      • First, celebrate the useful new functionality—Swift Testing support for macro testing!
      • Then, explain the known caveat(s). The main one being that if you have a package dependency on Swift Testing, you upgrade to this (major) version of SwiftSyntax, and you attempt to validate macros from a Swift Testing test, then it's possible failures may not be surfaced correctly.
      • Link to my new document in the Swift Testing repo about this topic, and emphasize that it's due to using a non-built in copy.
      • Offer two solutions for anyone affected by this:
        • Simplest: Remove the package dependency on Swift Testing and switch to using a built-in copy, as the document recommends.
        • Otherwise, switch from importing SwiftSyntaxMacrosTestSupport to SwiftSyntaxMacrosGenericTestSupport and specify your own failureHandler: closure in which you call Issue.record() yourself. That should work even when using a package copy of Swift Testing.
  4. Later, in a subsequent PR, adjust this logic for Swift Testing interoperability
    • If/when the pitched Interoperability feature arrives, assertMacroExpansion() will need to be adjusted to avoid redundant test issues being recorded due to it calling both Issue.record() and XCTFail().
    • To avoid that, at that point in time assertMacroExpansion() can be modified to stop calling XCTFail() starting in the Swift version where interoperability is first enabled.
    • (The call to XCTFail() shouldn't be removed altogether though, since SwiftSyntax could presumably still be built and used from older Swift versions.)

I've assembled a branch (atop this PR branch) with my suggested code changes for this PR in a commit you can see here: stmontgomery@c772dc6. Thoughts?

@stmontgomery
Copy link
Copy Markdown
Contributor

There are two other points above I wanted to respond to:

While the interoperability will allow the existing XCTest based method to work. It won't work with the strict mode which seems limiting.

Building on what @jerryjrchen said, the semantics of strict mode will be that in any situation where Swift Testing offers an API, that should be preferred over the analogous XCTest variant. It does not mandate that you must only call the APIs corresponding to the active test framework, in case that's something folks are concerned about here. I do think/hope strict mode can eventually be used in conjunction with this work, if end users choose to enable it.

We must not cause build failures for teams using Swift Testing as a package.

I recognize that this plan does have the potential to break users who have a package dependency on Swift Testing. I suspect the practical impact of that is likely small due to few users relying on it, and I think the tradeoff here is very worthwhile since we'll be delivering a frequently requested feature. The document I wrote about distribution locations of Swift Testing was intended to help clarify our recommendations regarding use of the package copy, and frankly, to discourage its use more strongly than we have in the past.

This change will roll out in a new major tag of SwiftSyntax, meaning (per SemVer) it will be an explicit action users must take so the timing of the upgrade will be in their control. And I believe we can mitigate and help users make the transition gracefully with a clearly-worded release note (discussed above).

@0xTim
Copy link
Copy Markdown
Member Author

0xTim commented Feb 3, 2026

@stmontgomery looks good to me - happy to pull those changes into my branch as I have done (just some extra changes to add if you agree around the can import stuff and expanding the test info) or have you submit the PR

@rintaro
Copy link
Copy Markdown
Member

rintaro commented Feb 6, 2026

Sorry I haven't been following this PR, so I may be missing some context.

FWIW, the original intent behind the current structure was to take advantage of cross-import overlays once it's accepted and SwiftPM supports them.
The idea was:

  • Rename the current XCTest-based SwiftSyntaxMacrosTestSupport to SwiftSyntaxMacrosTestSupport_XCTest
  • Rename the current SwiftSyntaxMacrosGenericTestSupport to SwiftSyntaxMacrosTestSupport
  • Introduce SwiftSyntaxMacrosTestSupport_SwiftTesting, which provides the swift-testing implementation of assertMacroExpansion()
    With that in place, users could simply write:
import SwiftSyntaxMacrosTestSupport
import SwiftTesting

and have SwiftSyntaxMacrosTestSupport_SwiftTesting picked up automatically via cross-import overlays.

That's said I'm a bit skeptical that cross-import overlays will be available in the near future. Also, as I understand it, we likely can't utilize Package traits because we treat XCTest and Testing as system libraries. Given that, plus the direction of the interoperability work, this PR (and the plan from @stmontgomery) seems to be the right direction.

A couple of questions:

Modify assertMacroExpansion() so that it calls both Issue.record() (Swift Testing) and XCTFail() (XCTest), one after the other.

Just to confirm, this is safe because XCTAssert failure in a Swift Testing test or an #expect failure in an XCTest test is silently ignored (from the interoperability proposal)?

What would happen after the interoperability is implemented? Would a failure be reported twice? Or the interoperability feature would consolidate the duplicated failure into one?

if you have a package dependency on Swift Testing, you upgrade to this (major) version of SwiftSyntax, and you attempt to validate macros from a Swift Testing test, then it's possible failures may not be surfaced correctly.

Could you elaborate on this? Any example that would break with this change? I don't believe a simple setup like this would work even with the current situation.

import SwiftSyntaxMacroTestSupport
import Testing

@Test func testMyMacro() {
  assertMacroExpansion(...)
}

Comment on lines +40 to +43
/// - Warning: If you call this function inside a Swift Testing test case,
/// the test will report no assertion failures even if the test fails.
/// Use ``expectMacroExpansion(_:expandedSource:diagnostics:macroSpecs:applyFixIts:fixedSource:testModuleName:testFileName:indentationWidth:buildConfiguration:fileID:file:line:column:)`` instead.
///
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (and the one on the another function) isn't true now.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since which Swift version? Because it's definitely the case in 6.2.3

Copy link
Copy Markdown
Member

@rintaro rintaro Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant after your latest change. You're not even adding expectMacroExpansion anymore. :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, very good point!

@rintaro
Copy link
Copy Markdown
Member

rintaro commented Feb 6, 2026

Could you elaborate on this? Any example that would break with this change? I don't believe a simple setup like this would work even with the current situation.

Ah, I see it now. SwiftSytaxMacrosTestingSupport might be built against the toolchain Testing, but the clients will be built against the just-built Testing because they have the dependencies. And that will cause the issue.

@0xTim
Copy link
Copy Markdown
Member Author

0xTim commented Feb 27, 2026

Anything I can do to push this along?

@grynspan
Copy link
Copy Markdown
Contributor

@0xTim
Copy link
Copy Markdown
Member Author

0xTim commented Feb 27, 2026

Right but that's targeting 6.4, assuming it's gets accepted (which for the record I'm in favour of).

As far as I can tell there's nothing stopping this being merged and it could, if there was the desire land in 6.3 and not force people to hit this issue for another 6 months. This PR has already been open for 3 months. It would be good to fix it sooner rather than later.

@grynspan
Copy link
Copy Markdown
Contributor

The review period is only a couple of weeks. Let's at least wait that out.

@stmontgomery
Copy link
Copy Markdown
Contributor

Hey @rintaro ,

A couple of questions:

Modify assertMacroExpansion() so that it calls both Issue.record() (Swift Testing) and XCTFail() (XCTest), one after the other.

Just to confirm, this is safe because XCTAssert failure in a Swift Testing test or an #expect failure in an XCTest test is silently ignored (from the interoperability proposal)?

Correct, that's the behavior prior to the interoperability feature.

What would happen after the interoperability is implemented? Would a failure be reported twice? Or the interoperability feature would consolidate the duplicated failure into one?

I think what we should do is proactively land this change with the call to XCTFail() guarded by #if compiler(<6.4) in anticipation of the interoperability feature being included in 6.4.

As of this writing no outcome has been announced for ST-0021, so we shouldn't actually land this PR with that guard in place until that happens to be safe. But assuming it does get accepted soon, in my opinion this PR could be landed and it could be included in 6.3 too. And that would mean that even when using a 6.3-series tag of SwiftSyntax with a future 6.4 toolchain which has Interoperability, things would work correctly and test failures would not be reported twice (redundantly).

if you have a package dependency on Swift Testing, you upgrade to this (major) version of SwiftSyntax, and you attempt to validate macros from a Swift Testing test, then it's possible failures may not be surfaced correctly.

Could you elaborate on this? Any example that would break with this change? I don't believe a simple setup like this would work even with the current situation.

import SwiftSyntaxMacroTestSupport
import Testing

@Test func testMyMacro() {
  assertMacroExpansion(...)
}

Here's an example to illustrate what I mean. Let's assume for the sake of discussion we move forward with this SwiftSyntax PR. Imagine a user has a package which has dependencies on both the swift-syntax and swift-testing packages — that is, they choose not to use the copy of Swift Testing included in the toolchain, but use its package copy instead. In that scenario, code in swift-syntax will now have import Testing and call its APIs. However, the swift-syntax package intentionally does not declare a package dependency on the swift-testing package itself; it just assumes that it will use the copy of that module from the toolchain. So it's an ambiguous dependency.

Because of that, the order the targets in the graph build in could lead to nondeterministic behavior. It's possible that the swift-syntax modules which import Testing may build before swift-testing package and thus, those swift-syntax modules will import and link the toolchain copy of libTesting. Or it's possible the order will be inverted and everything will link to the package copy of swift-testing. If the former happens, SwiftSyntax will record issues but do so against a copy of the testing library which is different than the one the end users' tests are using, so the issues will be silently ignored. But again, there are several potential outcomes and the whole situation is nondeterministic.

I lay all this out not to worry people; I fully support the plan above. I'm just trying to illustrate the known pitfalls of using a package copy of Swift Testing, and that's why I wrote a documentation article trying to discourage it. I do not think this concern should prevent us from proceeding with landing this PR.

@rintaro
Copy link
Copy Markdown
Member

rintaro commented Mar 14, 2026

I think what we should do is proactively land this change with the call to XCTFail() guarded by #if compiler(<6.4) in anticipation of the interoperability feature being included in 6.4.

This sounds good to me 👍

@0xTim
Copy link
Copy Markdown
Member Author

0xTim commented Mar 24, 2026

Ping again on this @stmontgomery given the timescales

@0xTim
Copy link
Copy Markdown
Member Author

0xTim commented Apr 12, 2026

Closing as there seems to be no desire to merge

@0xTim 0xTim closed this Apr 12, 2026
@grynspan
Copy link
Copy Markdown
Contributor

Reopening. Yes, there is, I promise!

@grynspan grynspan reopened this Apr 13, 2026
@grynspan grynspan requested a review from a team as a code owner April 13, 2026 00:28
…ng propagated, an update to a DocC symbol reference, and remove unnecessary parens from #if
@stmontgomery
Copy link
Copy Markdown
Contributor

I spent some time on this last night and wanted to share an update. My proposal for how to proceed with this had been the following:

  1. Fix a couple small things I saw in this PR while reviewing it closely (fileID: and column: args not being propagated, and an update to a DocC symbol reference).
  2. Guard the call to XCTFail() with #if compiler(<6.4), so that in toolchains where the ST-0021: Interoperability feature is enabled, only Issue.record() will be called.
  3. Test that using both 6.3 and 6.4 snapshot toolchains, to confirm both the pre- and post-interoperability scenarios work as expected.
  4. Once qualified, merge this PR into main, and then request to cherry-pick it into the release/6.3 branch for an upcoming 6.3.x release.

I pushed the changes for 1 directly to this PR branch since they're trivial (fe826cf).

However, after making the change in 2 and qualifying them (3) in a 6.4 snapshot toolchain I realized there's a problem with this rollout plan: the default interop mode. The mode is limited in the 6.4 toolchains, except for packages whose tools version is set to 6.4 or later (in which case it will be complete). In limited mode, issues propagated across the interop boundary are recorded with warning severity, meaning they don't cause a test failure. In practice, what this means is that when we initially roll this change out, for existing XCTests which use assertMacroExpansion(), if the assertion fails when using a 6.4 toolchain, Issue.record() will be called but it will only record an XCTest warning, and the test status will be Success instead of Failure.

I'm pretty sure that isn't the experience we want and that would be considered a regression; we need the final XCTest status to be a true Failure in that situation. So I also experimented with some workarounds and I do have one potentially viable alternative which keys off of both the interop environment variable and Test.current: see stmontgomery@6e07d36. However, I'm beginning to consider some Swift Testing enhancements to streamline this workflow and reduce the need for this kind of workaround. Still figuring that out, and I'd like to talk with @jerryjrchen and @grynspan about it, but wanted to give that update. I personally still want to find a way to get some version of this into a 6.3-series release if possible.

@FranzBusch
Copy link
Copy Markdown
Member

However, I'm beginning to consider some Swift Testing enhancements to streamline this workflow and reduce the need for this kind of workaround. Still figuring that out, and I'd like to talk with @jerryjrchen and @grynspan about it, but wanted to give that update. I personally still want to find a way to get some version of this into a 6.3-series release if possible.

Thanks for sharing all of this. Just to take a step back what if we consider a previous alternative again - Introduce an entirely new method that is only using Testing expectation APIs so we don't try to make a single method work in all contexts.

@stmontgomery
Copy link
Copy Markdown
Contributor

Thanks for sharing all of this. Just to take a step back what if we consider a previous alternative again - Introduce an entirely new method that is only using Testing expectation APIs so we don't try to make a single method work in all contexts.

Yes, I believe that would work. (We should test it though of course, including in a few interoperability scenarios, to confirm.) Part of why I've been getting involved and experimenting in this space is because I suspect the transition challenges we're observing here aren't unique to SwiftSyntax; other packages which vend test-helper libraries will encounter them too. So I've been looking for ways to avoid the need for clients to introduce "parallel" sets of helper functions (one for XCTest, one for Swift Testing) since it can often be difficult to know which is the correct one to call—especially if the helper is within a wrapper and isn't called directly from a "leaf node" test function. So while that approach may work for SwiftSyntax specifically, for other use cases that approach might be problematic.

Still, it's worth considering (again) in this PR, given that most macro tests do call this utility directly from the test function. Supposing we did take that approach, and introduced a new "expectMacroExpansion()" function, that would support interoperability too of course. So once it existed, it would be usable from XCTests too (with the severity caveat above), and over time as ST-0021 phases in and peoples' interop modes transition from limited to complete, that newer wrapper could become the "primary" one, and eventually assertMacroExpansion() could be deprecated outright.

@FranzBusch
Copy link
Copy Markdown
Member

Still, it's worth considering (again) in this PR, given that most macro tests do call this utility directly from the test function. Supposing we did take that approach, and introduced a new "expectMacroExpansion()" function, that would support interoperability too of course. So once it existed, it would be usable from XCTests too (with the severity caveat above), and over time as ST-0021 phases in and peoples' interop modes transition from limited to complete, that newer wrapper could become the "primary" one, and eventually assertMacroExpansion() could be deprecated outright.

Right. I would really like to take the next step as something that we can actually land. An expectMacroExpansion method seems reasonable to me and we can clearly document when to use what. It also provides a nice migration path for developers. Not as nice as automatic interop but it exists.

@grynspan
Copy link
Copy Markdown
Contributor

The reason we didn't pursue that solution already is that library authors can't tell which function to call on behalf of their own calers, and there's no reliable XCTest.isRunning- or Testing.isRunning-style API (for which we have had a bunch of different PRs and issues opened over the last year or so).

@0xTim
Copy link
Copy Markdown
Member Author

0xTim commented Apr 14, 2026

The reason we didn't pursue that solution already is that library authors can't tell which function to call on behalf of their own calers, and there's no reliable XCTest.isRunning- or Testing.isRunning-style API (for which we have had a bunch of different PRs and issues opened over the last year or so).

If we expose an expectMacroExpansion and document it's only expected to be used from Swift Testing, that would be a suitable compromise until the XCTest/Swift Testing interop stuff lands in 6.4 right? Then at that point, we can add support for that in the function and adjust the comment

@stmontgomery
Copy link
Copy Markdown
Contributor

I had a chat with @jerryjrchen and recently, he independently discovered the limitation I described above. He's preparing an amendment to ST-0021 with a change meant to help solve this. And if that proceeds, it means the original plan I outlined above would once again be viable, and we wouldn't need to introduce a new function here. The amendment would likely be considered for 6.4 as well, which means we could still land this change relatively quickly in a 6.3.x tag as previously planned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants