Introduce annotation to disable test based on active Spring profile [SPR-11677]

[spring-projects-issues]

Torsten Krah opened SPR-11677 and commented

I've looked at the @IfProfileValue and @ActiveProfiles annotations, but neither does what I am searching for.

We are using the "spring.profiles.active" system property to activate different profiles, even at the test level -- e.g. "h2, default" or something else.

We have tests which are only for Oracle and should therefore only be run if the "oracle" profile is active, but I've found no way to express this on the test class itself.

A dedicated annotation would be nice to support this.

---

Affects: 4.0.3

Issue Links:

• Decide what to do with @IfProfileValue [SPR-7754] #12410 Decide what to do with @IfProfileValue
• Introduce strategy for determining if a profile value is enabled for a particular test environment [SPR-4862] #9538 Introduce strategy for determining if a profile value is enabled for a particular test environment
• Allow overriding @ActiveProfiles in test classes with system property [SPR-8982] #13622 Allow overriding @ActiveProfiles in test classes with system property
• SystemProfileValueSource is not very compatible with the new 3.1 default system property profiles [SPR-8985] #13625 SystemProfileValueSource is not very compatible with the new 3.1 default system property profiles

8 votes, 8 watchers

[spring-projects-issues]

Sam Brannen commented

Hi Torsten,

I've linked several related issues: #9538, #13622, #13625, #12410.

Can you please review them to see if your needs would be addressed by one of them?

Thanks,

Sam

[spring-projects-issues]

Torsten Krah commented

If i read all of the issues right, they are about setting the profile in a test, i did not found a hint how to actually trigger the test class at all based on the active profile (which i did set already via System property when running the test suite @jenkins) - like @Ignore on the class or method - something like @RunWithSpringProfile("h2", "default") - or something.
#9538 seems to be the same direction but has no solution yet. Feel free to correct me if i missed some comment at one task you've linked.

Thanks, Torsten.

[spring-projects-issues]

Christoph Strobl commented

Hi,

We've had similar requirements when testing against different versions. Basically we created a custom JUnit @Rule to disable tests for features only available in Redis 2.8 when running against 2.6.

Let me outline what I am thinking of in terms of dealing with spring profiles.

I've added a sample below basically adding a new annotation IfSpringProfileActive holding the desired profile name. The SpringProfileRule checks for the presence of IfSpringProfileActive, loads the current Environment if not already set, and verifies that the desired profile is currently active.

There's SpringRuleTest at the end of the sample to show what it does.

/**
 * Annotation holding name of desired profile used to mark methods from being potentially excluded. 
 */
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface IfSpringProfileActive {

	String value();
}

/**
 * TestRule verifying Tests marked with IfSpringProfileActive are verified against currently active spring profile
 */
public class SpringProfileRule implements TestRule {

	private Environment env;

	private SpringProfileRule(Environment env) {
		this.env = env;
	}

	public static SpringProfileRule forSpringJunitClassRunner() {
		return new SpringProfileRule(null);
	}

	public static SpringProfileRule forEnvironment(Environment env) {
		return new SpringProfileRule(env);
	}

	@Override
	public Statement apply(final Statement base, Description description) {

		IfSpringProfileActive profileValue = description.getAnnotation(IfSpringProfileActive.class);
		final String requiredProfile = profileValue != null ? profileValue.value() : null;

		return new Statement() {

			@Override
			public void evaluate() throws Throwable {

				if (StringUtils.hasText(requiredProfile)) {
					initEnvironmentWhenNotSet(base);
					verify(requiredProfile);
				}

				base.evaluate();
			}
		};

	}

	protected void initEnvironmentWhenNotSet(Statement base) {

		if (env == null && base instanceof RunAfterTestMethodCallbacks) {

			// there should be a better way of doing this...
			TestContextManager contextManager = (TestContextManager) new DirectFieldAccessor(base)
					.getPropertyValue("testContextManager");
			TestContext testContext = (TestContext) new DirectFieldAccessor(contextManager).getPropertyValue("testContext");
			env = testContext.getApplicationContext().getEnvironment();
		}
	}

	protected void verify(String requiredProfile) throws Throwable {

		if (StringUtils.hasText(requiredProfile)) {
			if (!env.acceptsProfiles(requiredProfile)) {
				throw new AssumptionViolatedException(String.format("Profile %s is currently not active", requiredProfile));
			}
		}
	}

}

/**
 * A quick test to show that it works
 */
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration
@ActiveProfiles(profiles = "oracle")
public class SpringRuleTest {

	@Autowired String foo;

	public @Rule SpringProfileRule rule = SpringProfileRule.forSpringJunitClassRunner();

	@Configuration
	static class Config {

		@Bean
		String foo() {
			return "foo";
		}
	}

	@Test
	public void shouldAlwaysRun() {
		System.out.println("Run no matter what!");
	}

	@Test
	@IfSpringProfileActive("oracle")
	public void shouldOnlyRunWhenActiveProfileSetToOracle() {
		System.out.println("Runnig on oracle.");
	}

	@Test
	@IfSpringProfileActive("not-oracle")
	public void shouldBeIgnoredIfActiveProfileSetToOracle() {
		System.out.println("Oracle should not see me.");
	}

}

[spring-projects-issues]

Caleb Cushing commented

Why not just allow test methods and test classes to be annotated with @Profile and skip them based on the same rules beans would otherwise not be processed with?

This is the most obvious solution to me, and is the first solution I reached for.

[tauinger-de]

As mentioned above annotating test classes and methods with @Profile is a possible solution and IMHO the most intuitive way. Basically I did that without thinking and was wondering why it doesn't work.

[sbrannen]

Please note that it is already possible to enable/disable a test class or test method via a SpEL expression when using JUnit Jupiter (from JUnit 5).

Inspired by the test case supplied by @christophstrobl, I put together the following example.

import org.junit.jupiter.api.Test;
import org.springframework.context.annotation.Configuration;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.junit.jupiter.EnabledIf;
import org.springframework.test.context.junit.jupiter.SpringJUnitConfig;

@SpringJUnitConfig
@ActiveProfiles("oracle")
class SpringProfileTests {

	@Test
	@EnabledIf(expression = "#{environment.matchesProfiles('oracle')}", loadContext = true)
	void shouldOnlyRunWhenActiveProfileSetToOracle() {
		System.out.println("Runnig on oracle.");
	}

	@Test
	@EnabledIf(expression = "#{environment.matchesProfiles('!oracle')}", loadContext = true)
	// @DisabledIf(expression = "#{environment.matchesProfiles('oracle')}", loadContext = true)
	void shouldBeIgnoredIfActiveProfileSetToOracle() {
		throw new RuntimeException("Oracle should not see me.");
	}

	@Configuration
	static class Config { /* no beans */ }
}

The test class passes as-is, but if you set the active spring profile to not-oracle then shouldOnlyRunWhenActiveProfileSetToOracle() will be disabled, and shouldBeIgnoredIfActiveProfileSetToOracle() will be enabled and fail.

---

The key thing to note is the use of the @EnabledIf/@DisabledIf annotations from spring-test (not the @EnabledIf/@DisabledIf annotations from junit-jupiter-api).

@EnabledIf(expression = "#{environment.matchesProfiles('oracle')}", loadContext = true)

Another very important thing to note is that it only works with loadContext = true, and that means that the test's ApplicationContext will be eagerly loaded even if it's potentially never needed.

The latter is a downside to any approach I can think of to reliably "disable a test based on an active Spring profile".

Using @Profile on test classes or test methods -- as suggested in various comments in this issue -- would have the same downside.