I recently retweeted the following:
+1— Chris Richardson (@crichardson) June 17, 2019
Dependencies can/should run in containers for consistency but development of the service should happen locally. https://t.co/xLUrsTxuK8
This must have resonated with some people because it was retweeted and liked quite a few times.
However, not all reactions were positive:
Actually, one of the biggest killers of developer productivity is the inconsistency of local development environments and/or difficulty of setting one up. So this statement is nothing but a wild opinion.— Irakli Nadareishvili (@inadarei) June 17, 2019
As a result, I thought I’d write this post to expand on my original comment.
Traditionally, one major challenge with setting up a developer’s environment is installing and maintaining all the necessary infrastructure services, such as databases and message brokers. Some infrastructure services are tricky to install. Keeping every developer’s environment up to date and consistent is error-prone and time consuming. Fortunately, Docker-based tools, such as Docker Compose or Kubernetes, provide a very convenient infrastructure-as-code solution.
A service’s source code can simply include one or more configuration files, such as a
docker-compose.yml file or Kubernetes YAML files, that define the service’s infrastructure dependencies.
For example, my book’s FTGO application has a
docker-compose.yml file in the root directory.
These configuration files are versioned, which guarantees that each developer has the correct versions of the infrastructure services.
And, starting the services is simply a matter of running
docker-compose up -d or a
kubectl apply -f ..., which will automatically download and run the service containers.
Docker/Kubernetes is the last piece of infrastructure that you need to install.
Let’s now look at when you might package a service as a Docker container during development and testing.
The test pyramid is a good guide for the kinds of tests that you need to write. At the base of the pyramid are the fast, simple, and reliable unit tests. At the top of the pyramid are the slow, complex, and brittle end-to-end tests. Like the USDA food pyramid, although more useful and less controversial, the test pyramid describes the relative proportions of each type of test.
The different layers of the pyramid are as follows:
The key idea of the test pyramid is that as we move up the pyramid we should write fewer and fewer tests. We should write lots of unit tests and very few end-to-end tests. Let’s look at which kinds of tests for a service that can use Docker.
The most fundamental developer workflow is the edit-build-run-debug loop. You make a change, build the code, run some tests (the Compile Suite), debug it, repeat. If you are doing TDD, this loop begins with writing a test. In a modern IDE, you can build the code and run the tests with a single keystroke.
It’s important for this loop to be as fast as possible. In particular, you want to minimize the build time and the test execution time. You need to be as far down the test pyramid as possible. You want to avoid building and testing the service as a whole. And, you certainly shouldn’t build Docker containers.
Ideally, you should only run unit tests during the edit-build-run-debug loop since those execute extremely quickly.
They typically use test doubles to avoid interacting with any infrastructure.
For example, in the FTGO application, the
OrderTest is a unit test class for the
Order aggregate, and
OrderControllerTest is a unit test for the
If, however, you are developing an adapter class, which interacts with a message broker or the database then you clearly need more than unit tests.
Fortunately, this doesn’t mean building and testing the entire service.
Instead, you can run integration tests that verify the behavior of just the adapter class.
There is still no reason to build and run a Docker container.
In the FTGO application, for example,
OrderJpaTest is an integration test for the
OrderRepository JPA repository class.
It’s only when you get to the third level of pyramid - the component tests - does it make sense to run the service-under-development as a Docker container. A component test verifies the behavior of a service by invoking its API (or its UI) It typically tests the service in isolation by using test doubles for the services it depends on.
There are two kinds of component tests: in-process or out of process.
An in-process component test, runs the service in the test’s process using in-memory test doubles for its dependencies.
It might even use an in-memory database for the service’s database.
In-process component tests are typically straightforward write.
For example, Spring Boot’s
@SpringBootTest annotation makes it very easy to write a test that starts a service on a random port.
But in-process component tests have the downside of not testing the deployable service.
It’s often more effective to write out of process component tests. An out of process component test packages the service in a production-ready format and runs it as a separate process. The service uses a real database and message broker but its other (application service) dependencies are replaced test doubles.
If you are using Docker in a production, it makes sense for the test to package the service as a Docker container image. Finally, there is a step of development workflow that involves building a Docker image for the service!
The FTGO application has an example of this approach to writing component tests.
The following diagram shows the design of the out of process component tests for the
The tests are implemented by the OrderServiceComponentTest class.
They use Docker Compose (via the Gradle Docker Compose Plugin) to build and run the
Order Service and its infrastructure dependencies, which include Apache Kafka and MySQL.
Order Service invokes its service dependencies using request/asynchronous response over Apache Kafka, the test doubles for those dependencies subscribe to Apache Kafka.
End-to-end tests verify the behavior of the entire application. They deploy the application in a test environment and invoke either its API or UI. It makes sense for the tests to use the same deployment mechanism, such as a Docker orchestration framework, that is used in production.
Microservices.io is brought to you by Chris Richardson. Experienced software architect, author of POJOs in Action, the creator of the original CloudFoundry.com, and the author of Microservices patterns.
Chris helps clients around the world adopt the microservice architecture through consulting engagements, and training workshops.
Chris teaches comprehensive workshops for architects and developers that will enable your organization use microservices effectively.
Avoid the pitfalls of adopting microservices and learn essential topics, such as service decomposition and design and how to refactor a monolith to microservices.Learn more
Chris offers numerous other resources for learning the microservice architecture.
Want to see an example? Check out Chris Richardson's example applications. See code
Got a specific microservice architecture-related question? For example:
Consider signing up for a two hour, highly focussed, consulting session.
My virtual bootcamp, distributed data patterns in a microservice architecture, is now open for enrollment!
It covers the key distributed data management patterns including Saga, API Composition, and CQRS.
It consists of video lectures, code labs, and a weekly ask-me-anything video conference repeated in multiple timezones.
The regular price is $395/person but use coupon MECNPWNR to sign up for $120 (valid until May 16th, 2023). There are deeper discounts for buying multiple seats.
Take a look at my Manning LiveProject that teaches you how to develop a service template and microservice chassis.
Engage Chris to create a microservices adoption roadmap and help you define your microservice architecture,
Use the Eventuate.io platform to tackle distributed data management challenges in your microservices architecture.
Eventuate is Chris's latest startup. It makes it easy to use the Saga pattern to manage transactions and the CQRS pattern to implement queries.
Engage Chris to conduct an architectural assessment.
Note: tagging is work-in-process
anti-patterns · application api · application architecture · architecting · architecture documentation · assemblage · beer · containers · dark energy and dark matter · deployment · design-time coupling · development · devops · docker · eventuate platform · glossary · hexagonal architecture · implementing commands · implementing queries · inter-service communication · kubernetes · loose coupling · microservice architecture · microservice chassis · microservices adoption · microservicesio updates · multi-architecture docker images · observability · pattern · refactoring to microservices · resilience · sagas · security · service api · service collaboration · service design · service discovery · service granularity · service template · software delivery metrics · success triangle · tacos · team topologies · transaction management · transactional messaging