Dynamic Tests in JUnit 5: Generating Tests at Runtime

In many testing scenarios, the exact number of tests you need isn’t known at compile time. For example, validating data sets, verifying configurations, or testing against API responses may require generating tests dynamically at runtime. JUnit 5 introduces the @TestFactory annotation, enabling developers to create dynamic tests programmatically.

This tutorial will cover how to use dynamic tests in JUnit 5, integrate them with Java Streams, apply them in real-world scenarios like mocking and Testcontainers, and follow best practices for maintainability.

---

Why Dynamic Tests?

• Flexibility: Generate tests based on data, conditions, or runtime logic.
• Coverage: Test multiple inputs without hardcoding methods.
• Maintainability: Avoid repetitive boilerplate test methods.
• Real-World Fit: Ideal for scenarios like validating datasets or running against multiple environments.

Think of dynamic tests as on-demand tickets — you don’t print them until you know who’s attending.

---

Creating Dynamic Tests with @TestFactory

The @TestFactory annotation allows defining a factory method that generates tests dynamically.

import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;

import java.util.Arrays;
import java.util.Collection;

import static org.junit.jupiter.api.Assertions.assertTrue;

class DynamicTestExample {

    @TestFactory
    Collection<DynamicTest> dynamicTestsFromCollection() {
        return Arrays.asList(
                DynamicTest.dynamicTest("Test 1", () -> assertTrue(2 > 1)),
                DynamicTest.dynamicTest("Test 2", () -> assertTrue("hello".startsWith("h")))
        );
    }
}

Here, two tests are created dynamically at runtime.

---

Generating Tests with Streams

You can use Streams to create scalable dynamic tests.

import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;

import java.util.stream.IntStream;
import java.util.stream.Stream;

import static org.junit.jupiter.api.Assertions.assertTrue;

class StreamDynamicTest {

    @TestFactory
    Stream<DynamicTest> dynamicTestsFromStream() {
        return IntStream.range(1, 6)
                .mapToObj(n -> DynamicTest.dynamicTest("Check number " + n,
                        () -> assertTrue(n > 0)));
    }
}

This generates five tests dynamically for numbers 1 through 5.

---

Dynamic Tests with Assertions

You can integrate different assertions for validation.

import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;

import java.util.List;

import static org.junit.jupiter.api.Assertions.assertEquals;

class AssertionDynamicTest {

    @TestFactory
    List<DynamicTest> dynamicStringTests() {
        List<String> inputs = List.of("apple", "banana", "cherry");

        return inputs.stream()
                .map(input -> DynamicTest.dynamicTest("Testing length of " + input,
                        () -> assertEquals(5, input.length())))
                .toList();
    }
}

In this example, not all tests will pass, which highlights how dynamic tests catch unexpected data issues.

---

Dynamic Tests with Mockito

Mockito can be combined with dynamic tests to simulate varying inputs.

import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;
import org.mockito.Mockito;

import java.util.List;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.Mockito.*;

class MockitoDynamicTest {

    interface Calculator {
        int add(int a, int b);
    }

    @TestFactory
    List<DynamicTest> testCalculatorWithMocks() {
        Calculator calculator = mock(Calculator.class);
        when(calculator.add(anyInt(), anyInt())).thenAnswer(inv -> {
            int a = inv.getArgument(0);
            int b = inv.getArgument(1);
            return a + b;
        });

        List<int[]> testCases = List.of(new int[]{1, 2}, new int[]{5, 7}, new int[]{10, -3});

        return testCases.stream()
                .map(tc -> DynamicTest.dynamicTest("Add " + tc[0] + " + " + tc[1],
                        () -> assertEquals(tc[0] + tc[1], calculator.add(tc[0], tc[1]))))
                .toList();
    }
}

---

Dynamic Tests with Testcontainers

Dynamic tests can validate environments created with Testcontainers.

import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.TestFactory;
import org.testcontainers.containers.PostgreSQLContainer;

import java.util.List;

class TestcontainersDynamicTest {

    @TestFactory
    List<DynamicTest> testMultipleDatabases() {
        List<String> versions = List.of("postgres:14", "postgres:15");

        return versions.stream()
                .map(version -> DynamicTest.dynamicTest("Test with " + version,
                        () -> {
                            try (PostgreSQLContainer<?> postgres = new PostgreSQLContainer<>(version)) {
                                postgres.start();
                                System.out.println("Running DB: " + postgres.getJdbcUrl());
                                assert postgres.isRunning();
                            }
                        }))
                .toList();
    }
}

Sample output:

Running DB: jdbc:postgresql://localhost:32789/test
Running DB: jdbc:postgresql://localhost:32790/test

---

Real-World Scenarios

1. Spring Boot Apps: Generate tests dynamically for multiple profiles or configs.
2. Microservices: Validate endpoints across various environments.
3. Legacy Codebases: Run dynamic validations on old datasets.
4. CI/CD Pipelines: Generate runtime tests for environment-specific conditions.

---

Best Practices

• Use dynamic tests for data-driven scenarios.
• Always log test names clearly for readability.
• Avoid excessive complexity in factories — keep them maintainable.
• Combine with mocks and containers for integration-level validation.
• Document data sources for reproducibility.

---

Version Tracker

• JUnit 4 → JUnit 5: Dynamic tests are a brand-new feature in JUnit 5.
• Mockito Updates: Inline mocking allows flexible test generation.
• Testcontainers Growth: Easier to validate multi-environment scenarios dynamically.

---

Conclusion & Key Takeaways

Dynamic tests in JUnit 5 provide runtime flexibility, allowing developers to generate tests on the fly for data-driven, environment-specific, or mock-based scenarios. They make test suites more powerful, maintainable, and CI/CD-ready.

Key Takeaways:

• Use @TestFactory to generate tests at runtime.
• Leverage Streams for scalable test generation.
• Combine with Mockito for service mocks and Testcontainers for databases.
• Ideal for microservices, CI/CD, and dynamic data validation.

---

FAQ

1. What is the difference between @Test and @TestFactory?
@Test defines static tests, while @TestFactory generates dynamic tests at runtime.

2. Can I use assertions in dynamic tests?
Yes, dynamic tests use the same assertion APIs as regular tests.

3. Do dynamic tests work with parameterized tests?
They are different — parameterized tests are predefined, dynamic tests are generated at runtime.

4. How do I debug failing dynamic tests?
Use clear display names to identify failing cases.

5. Can dynamic tests run in parallel?
Yes, depending on JUnit 5 configuration and test runners.

6. Do IDEs support dynamic tests?
Yes, IntelliJ IDEA, Eclipse, and build tools fully support them.

7. How do I use dynamic tests with Mockito?
Combine mocks with test data streams to validate different inputs.

8. Can Testcontainers be used with dynamic tests?
Yes, to test multiple containerized environments or versions.

9. Are dynamic tests suitable for CI/CD pipelines?
Yes, they allow environment-driven and data-driven validations in pipelines.

10. Should I migrate to JUnit 5 for dynamic tests?
Yes, this feature is unique to JUnit 5 and offers significant advantages.