Containers, the developer workflow and the test pyramid

I recently retweeted the following:

+1

Dependencies can/should run in containers for consistency but development of the service should happen locally. https://t.co/xLUrsTxuK8

— Chris Richardson (@crichardson) June 17, 2019

This must have resonated with some people because it was retweeted and liked quite a few times.

However, not all reactions were positive:

Actually, one of the biggest killers of developer productivity is the inconsistency of local development environments and/or difficulty of setting one up. So this statement is nothing but a wild opinion.

— Irakli Nadareishvili (@inadarei) June 17, 2019

As a result, I thought I’d write this post to expand on my original comment.

Summary:

• Use Docker (E.g. Docker Compose or Kubernetes) to reliably provision infrastructure services for a development environment
• Optimize the speed of the edit-build-run-debug cycle by testing as far down the test pyramid as possible
• Unit and integration tests should not run the service as Docker container
• Component tests are the lowest level of the test pyramid can run the service-under-development as Docker container

Docker is a great way to provide a standardized development environment

Traditionally, one major challenge with setting up a developer’s environment is installing and maintaining all the necessary infrastructure services, such as databases and message brokers. Some infrastructure services are tricky to install. Keeping every developer’s environment up to date and consistent is error-prone and time consuming. Fortunately, Docker-based tools, such as Docker Compose or Kubernetes, provide a very convenient infrastructure-as-code solution.

A service’s source code can simply include one or more configuration files, such as a docker-compose.yml file or Kubernetes YAML files, that define the service’s infrastructure dependencies. For example, my book’s FTGO application has a docker-compose.yml file in the root directory.

These configuration files are versioned, which guarantees that each developer has the correct versions of the infrastructure services. And, starting the services is simply a matter of running docker-compose up -d or a kubectl apply -f ..., which will automatically download and run the service containers. Docker/Kubernetes is the last piece of infrastructure that you need to install. Let’s now look at when you might package a service as a Docker container during development and testing.

The test pyramid

The test pyramid is a good guide for the kinds of tests that you need to write. At the base of the pyramid are the fast, simple, and reliable unit tests. At the top of the pyramid are the slow, complex, and brittle end-to-end tests. Like the USDA food pyramid, although more useful and less controversial, the test pyramid describes the relative proportions of each type of test.

The different layers of the pyramid are as follows:

• Unit tests – Test a small part of an service, such as a class
• Integration tests – Verify that a service can interact with infrastructure services, such as databases and other application services, by testing the service’s adapters
• Component tests – verify the behavior of an individual service
• End-to-end tests – verify the behavior of the entire application

The key idea of the test pyramid is that as we move up the pyramid we should write fewer and fewer tests. We should write lots of unit tests and very few end-to-end tests. Let’s look at which kinds of tests for a service that can use Docker.

Avoid building Docker images during your edit-compile-run-debug loop

The most fundamental developer workflow is the edit-build-run-debug loop. You make a change, build the code, run some tests (the Compile Suite), debug it, repeat. If you are doing TDD, this loop begins with writing a test. In a modern IDE, you can build the code and run the tests with a single keystroke.

It’s important for this loop to be as fast as possible. In particular, you want to minimize the build time and the test execution time. You need to be as far down the test pyramid as possible. You want to avoid building and testing the service as a whole. And, you certainly shouldn’t build Docker containers.

Ideally, you should only run unit tests during the edit-build-run-debug loop since those execute extremely quickly. They typically use test doubles to avoid interacting with any infrastructure. For example, in the FTGO application, the OrderTest is a unit test class for the Order aggregate, and OrderControllerTest is a unit test for the OrderController class.

If, however, you are developing an adapter class, which interacts with a message broker or the database then you clearly need more than unit tests. Fortunately, this doesn’t mean building and testing the entire service. Instead, you can run integration tests that verify the behavior of just the adapter class. There is still no reason to build and run a Docker container. In the FTGO application, for example, OrderJpaTest is an integration test for the OrderRepository JPA repository class.

Component tests can use Docker

It’s only when you get to the third level of pyramid - the component tests - does it make sense to run the service-under-development as a Docker container. A component test verifies the behavior of a service by invoking its API (or its UI) It typically tests the service in isolation by using test doubles for the services it depends on.

There are two kinds of component tests: in-process or out of process. An in-process component test, runs the service in the test’s process using in-memory test doubles for its dependencies. It might even use an in-memory database for the service’s database. In-process component tests are typically straightforward write. For example, Spring Boot’s @SpringBootTest annotation makes it very easy to write a test that starts a service on a random port. But in-process component tests have the downside of not testing the deployable service.

It’s often more effective to write out of process component tests. An out of process component test packages the service in a production-ready format and runs it as a separate process. The service uses a real database and message broker but its other (application service) dependencies are replaced test doubles.

If you are using Docker in a production, it makes sense for the test to package the service as a Docker container image. Finally, there is a step of development workflow that involves building a Docker image for the service!

The FTGO application has an example of this approach to writing component tests. The following diagram shows the design of the out of process component tests for the Order Service.

The tests are implemented by the OrderServiceComponentTest class. They use Docker Compose (via the Gradle Docker Compose Plugin) to build and run the Order Service and its infrastructure dependencies, which include Apache Kafka and MySQL. Since the Order Service invokes its service dependencies using request/asynchronous response over Apache Kafka, the test doubles for those dependencies subscribe to Apache Kafka.

End-to-end tests

End-to-end tests verify the behavior of the entire application. They deploy the application in a test environment and invoke either its API or UI. It makes sense for the tests to use the same deployment mechanism, such as a Docker orchestration framework, that is used in production.