Skip to main content

Test suites

The Testing section of the Temporal Application development guide describes the frameworks that facilitate Workflow and integration testing.

In the context of Temporal, you can create these types of automated tests:

  • End-to-end: Running a Temporal Server and Worker with all its Workflows and Activities; starting and interacting with Workflows from a Client.
  • Integration: Anything between end-to-end and unit testing.
    • Running Activities with mocked Context and other SDK imports (and usually network requests).
    • Running Workers with mock Activities, and using a Client to start Workflows.
    • Running Workflows with mocked SDK imports.
  • Unit: Running a piece of Workflow or Activity code (a function or method) and mocking any code it calls.

We generally recommend writing the majority of your tests as integration tests.

Because the test server supports skipping time, use the test server for both end-to-end and integration tests with Workers.

Test frameworks​

The Temporal Java SDK provides a test framework to facilitate Workflow unit and integration testing. The test framework provides a TestWorkflowEnvironment class which includes an in-memory implementation of the Temporal service that supports automatic time skipping. This allows you to easily test long-running Workflows in seconds, without having to change your Workflow code.

You can use the provided TestWorkflowEnvironment with a Java unit testing framework of your choice, such as JUnit.

Setup testing dependency​

To start using the Java SDK test framework, you need to add io.temporal:temporal-testing as a dependency to your project:

Apache Maven:

<dependency>
    <groupId>io.temporal</groupId>
    <artifactId>temporal-testing</artifactId>
    <version>1.17.0</version>
    <scope>test</scope>
</dependency>

Gradle Groovy DSL:

testImplementation ("io.temporal:temporal-testing:1.17.0")

Make sure to set the version that matches your dependency version of the Temporal Java SDK.

Sample unit tests​

The following code implements unit tests for the HelloActivity sample:

public class HelloActivityTest {

    private TestWorkflowEnvironment testEnv;
    private Worker worker;
    private WorkflowClient client;

    // Set up the test workflow environment
    @Before
    public void setUp() {
        testEnv = TestWorkflowEnvironment.newInstance();
        worker = testEnv.newWorker(TASK_QUEUE);
        // Register your workflow implementations
        worker.registerWorkflowImplementationTypes(GreetingWorkflowImpl.class);

        client = testEnv.getWorkflowClient();
    }

    // Clean up test environment after tests are completed
    @After
    public void tearDown() {
        testEnv.close();
    }

    @Test
    public void testActivityImpl() {
        // This uses the actual activity impl
        worker.registerActivitiesImplementations(new GreetingActivitiesImpl());

        // Start test environment
        testEnv.start();

        // Create the workflow stub
        GreetingWorkflow workflow =
                client.newWorkflowStub(
                        GreetingWorkflow.class, WorkflowOptions.newBuilder().setTaskQueue(TASK_QUEUE).build());

        // Execute our workflow waiting for it to complete
        String greeting = workflow.getGreeting("World");
        assertEquals("Hello World!", greeting);
    }
}

In cases where you do not wish to execute your actual Activity implementations during unit testing, you can use a framework such as Mockito to mock them.

The following code implements a unit test for the HelloActivity sample which shows how activities can be mocked:

public class HelloActivityTest {

    private TestWorkflowEnvironment testEnv;
    private Worker worker;
    private WorkflowClient client;

    // Set up the test workflow environment
    @Before
    public void setUp() {
        testEnv = TestWorkflowEnvironment.newInstance();
        worker = testEnv.newWorker(TASK_QUEUE);
        // Register your workflow implementations
        worker.registerWorkflowImplementationTypes(GreetingWorkflowImpl.class);

        client = testEnv.getWorkflowClient();
    }

    // Clean up test environment after tests are completed
    @After
    public void tearDown() {
        testEnv.close();
    }

    @Test
    public void testMockedActivity() {
        // Mock our workflow activity
        GreetingActivities activities = mock(GreetingActivities.class);
        when(activities.composeGreeting("Hello", "World")).thenReturn("Hello Mocked World!");
        worker.registerActivitiesImplementations(activities);

        // Start test environment
        testEnv.start();

        // Create the workflow stub
        GreetingWorkflow workflow =
                client.newWorkflowStub(
                        GreetingWorkflow.class, WorkflowOptions.newBuilder().setTaskQueue(TASK_QUEUE).build());

        // Execute our workflow waiting for it to complete
        String greeting = workflow.getGreeting("World");
        assertEquals("Hello Mocked World!", greeting);
    }
}

Testing with JUnit4​

For Junit4 tests, Temporal provides the TestWorkflowRule class which simplifies the Temporal test environment setup, as well as the creation and shutdown of Workflow Workers in your tests.

Make sure to set the version that matches your dependency version of the Temporal Java SDK.

We can now rewrite our above mentioned "HelloActivityTest" test class as follows:

public class HelloActivityJUnit4Test {
    @Rule
    public TestWorkflowRule testWorkflowRule =
            TestWorkflowRule.newBuilder()
                    .setWorkflowTypes(GreetingWorkflowImpl.class)
                    .setActivityImplementations(new GreetingActivitiesImpl())
                    .build();

    @Test
    public void testActivityImpl() {
        // Get a workflow stub using the same task queue the worker uses.
        GreetingWorkflow workflow =
                testWorkflowRule
                        .getWorkflowClient()
                        .newWorkflowStub(
                                GreetingWorkflow.class,
                                WorkflowOptions.newBuilder().setTaskQueue(testWorkflowRule.getTaskQueue()).build());
        // Execute a workflow waiting for it to complete.
        String greeting = workflow.getGreeting("World");
        assertEquals("Hello World!", greeting);

        testWorkflowRule.getTestEnvironment().shutdown();
    }
}

Testing with JUnit5​

For Junit5 tests, Temporal also provides the TestWorkflowExtension helper class. This class can be used to simplify the Temporal test environment setup as well as Workflow Worker startup and shutdowns.

To start using JUnit5 TestWorkflowExtension in your tests with Gradle, you need to enable capability [io.temporal:temporal-testing-junit5]:

Make sure to set the version that matches your dependency version of the Temporal Java SDK.

We can now use JUnit5 and rewrite our above mentioned "HelloActivityTest" test class as follows:

public class HelloActivityJUnit5Test {
    @RegisterExtension
    public static final TestWorkflowExtension testWorkflowExtension =
            TestWorkflowExtension.newBuilder()
                    .setWorkflowTypes(GreetingWorkflowImpl.class)
                    .setActivityImplementations(new GreetingActivitiesImpl())
                    .build();

    @Test
    public void testActivityImpl(
            TestWorkflowEnvironment testEnv, Worker worker, GreetingWorkflow workflow) {
        // Execute a workflow waiting for it to complete.
        String greeting = workflow.getGreeting("World");
        assertEquals("Hello World!", greeting);
    }
}

You can find all unit tests for the Temporal Java samples repository in its test package.

Test Activities​

Mocking isolates code undergoing testing so the focus remains on the code, and not on external dependencies or other state. You can test Activities using a mocked Activity environment.

This approach offers a way to mock the Activity context, listen to Heartbeats, and cancel the Activity. You test the Activity in isolation, calling it directly without needing to create a Worker to run it.

Temporal provides the TestActivityEnvironment and TestActivityExtension classes for testing Activities outside the scope of a Workflow. Testing Activities are similar to testing non-Temporal Java code.

For example, you can test an Activity for:

  • Exceptions thrown when invoking the Activity Execution.
  • Exceptions thrown when checking for the result of the Activity Execution.
  • Activity's return values. Check that the return value matches the expected value.

Run an Activity​

During isolation testing, if an Activity references its context, you'll need to mock that context. Mocked information stands in for the context, allowing you to focus your testing on the Activity's code.

Listen to Heartbeats​

Activities usually issue periodic Heartbeats, a feature that broadcasts recurring proof-of-life updates. Each ping shows that an Activity is making progress and the Worker hasn't crashed. Heartbeats may include details that report task progress in the event an Activity Worker crashes.

When testing Activities that support Heartbeats, make sure you can see those Heartbeats in your test code. Provide appropriate test coverage. This enables you to verify both the Heartbeat's content and behavior.

Cancel an Activity​

Activity cancellation lets Activities know they don't need to continue work and gives time for the Activity to clean up any resources it's created. You can cancel Java-based activities if they emit Heartbeats. To test an Activity that reacts to Cancellations, make sure that the Activity reacts correctly and cancels.

Testing Workflows​

How to mock Activities​

Mock the Activity invocation when unit testing your Workflows.

When integration testing Workflows with a Worker, you can mock Activities by providing mock Activity implementations to the Worker.

How to skip time​

Some long-running Workflows can persist for months or even years. Implementing the test framework allows your Workflow code to skip time and complete your tests in seconds rather than the Workflow's specified amount.

For example, if you have a Workflow sleep for a day, or have an Activity failure with a long retry interval, you don't need to wait the entire length of the sleep period to test whether the sleep function works. Instead, test the logic that happens after the sleep by skipping forward in time and complete your tests in a timely manner.

The test framework included in most SDKs is an in-memory implementation of Temporal Server that supports skipping time. Time is a global property of an instance of TestWorkflowEnvironment: skipping time (either automatically or manually) applies to all currently running tests. If you need different time behaviors for different tests, run your tests in a series or with separate instances of the test server. For example, you could run all tests with automatic time skipping in parallel, and then all tests with manual time skipping in series, and then all tests without time skipping in parallel.

Set up time skipping​

Learn to set up the time-skipping test framework in the SDK of your choice.

Skip time automatically​

You can skip time automatically in the SDK of your choice. Start a test server process that skips time as needed. For example, in the time-skipping mode, Timers, which include sleeps and conditional timeouts, are fast-forwarded except when Activities are running.

Skip time manually​

Learn to skip time manually in the SDK of your choice.

Skip time in Activities​

Learn to skip time in Activities in the SDK of your choice.

How to Replay a Workflow Execution​

Replay recreates the exact state of a Workflow Execution. You can replay a Workflow from the beginning of its Event History.

Replay succeeds only if the Workflow Definition is compatible with the provided history from a deterministic point of view.

When you test changes to your Workflow Definitions, we recommend doing the following as part of your CI checks:

  1. Determine which Workflow Types or Task Queues (or both) will be targeted by the Worker code under test.
  2. Download the Event Histories of a representative set of recent open and closed Workflows from each Task Queue, either programmatically using the SDK client or via the Temporal CLI.
  3. Run the Event Histories through replay.
  4. Fail CI if any error is encountered during replay.

The following are examples of fetching and replaying Event Histories:

To replay Workflow Executions, use the WorkflowReplayer class in the temporal-testing package.

In the following example, Event Histories are downloaded from the server, and then replayed. Note that this requires Advanced Visibility to be enabled.

// Note we assume you already have a WorkflowServiceStubs (`service`) and WorkflowClient (`client`)
// in scope.
ListWorkflowExecutionsRequest listWorkflowExecutionRequest =
    ListWorkflowExecutionsRequest.newBuilder()
        .setNamespace(client.getOptions().getNamespace())
        .setQuery("TaskQueue = 'mytaskqueue'")
        .build();
ListWorkflowExecutionsResponse listWorkflowExecutionsResponse =
    service.blockingStub().listWorkflowExecutions(listWorkflowExecutionRequest);
List<WorkflowExecutionHistory> histories =
    listWorkflowExecutionsResponse.getExecutionsList().stream()
        .map(
            (info) -> {
              GetWorkflowExecutionHistoryResponse weh =
                  service.blockingStub().getWorkflowExecutionHistory(
                      GetWorkflowExecutionHistoryRequest.newBuilder()
                          .setNamespace(testEnvironment.getNamespace())
                          .setExecution(info.getExecution())
                          .build());
              return new WorkflowExecutionHistory(
                  weh.getHistory(), info.getExecution().getWorkflowId());
            })
        .collect(Collectors.toList());

WorkflowReplayer.replayWorkflowExecutions(
    histories, true, WorkflowA.class, WorkflowB.class, WorkflowC.class);

In the next example, a single history is loaded from a JSON file on disk:

File file = new File("my_history.json");
WorkflowReplayer.replayWorkflowExecution(file, MyWorkflow.class);

In both examples, if Event History is non-deterministic, an error is thrown. You can choose to wait until all histories have been replayed with replayWorkflowExecutions by setting the failFast argument to false.