This is the first in a series of posts that expands on my recent MicroCPH talk on Managing data consistency in a microservice architecture using Sagas (slides, video).
The other posts in this series are:
A distinctive characteristic of the microservice architecture is that in order to ensure loose coupling each service’s data is private. Unlike in a monolithic application, you no longer have a single database that any module of the application can update. As a result, one of the key challenges that you will face is maintaining data consistency across services.
Consider, for example, the customers and orders example application from my presentation. It consists of two services:
Order Service creates an
Order it must ensure that there is sufficient credit available.
createOrder() command must update data in both the
Order Service and the
In a traditional application, you might consider using distributed transactions a.k.a. two phase commit (2PC). However, using 2PC is generally a bad idea a microservice architecture. It’s a form of synchronous communication that results in runtime coupling that significantly impacts the availability of an application.
The solution is to implement commands, such as
createOrder(), using a saga.
A saga is a sequence of local transactions in each of the participating services.
For example, here is the definition of the
Create Order Saga, which is initiated by the
The purpose of each step is as follows:
createPendingOrder()- create the
reserveCredit()- attempt to reserve credit
approveOrder()- change the state of the
rejectOrder()- change the state of the
The sequence for the happy path is as follows:
The sequence for the path when there is insufficient credit is as follows:
rejectOrder() command is an example of a compensating transaction.
Unlike ACID transactions, sagas cannot automatically undo changes made by previous steps since those changes are already committed.
Instead, you must write compensating transactions that explicitly undo those changes.
Each step of a saga that is followed by a step that can fail (for business reasons) must have a corresponding compensating transaction.
Create Order Saga,
createOrder() has the
rejectOrder() compensating transaction because the
reserveCredit() step can fail.
reserveCredit() step does not need a compensating transaction because the
approveOrder() step cannot fail.
approveOrder() step does not need a compensating transaction because it’s the last step of the saga.
You might be wondering why
createOrder() creates the order in a
PENDING state, which is then changed to
The use of a
PENDING state is an example of what is known as a semantic lock counter-measure.
It prevents another transaction/saga from updating the
Order while it is in the process of being created.
To see why this is necessary consider the following scenario where the
cancelOrder() command is invoked while the
Order is still being created:
|Create Order Saga||Cancel Order Saga|
In this scenario, the
cancelOrder() command changes the status of the order to
CANCELLED, and the
approveOrder() command overwrites that change by setting the status to
The customer would be quite surprised when the order is delivered!
PENDING state prevents this problem.
cancelOrder() command will only cancel an
Order if its state is
If the state is
cancelOrder() returns an error to the client indicating that it should try again later.
The semantic lock counter-measure is a kind of application-level locking.
As I describe in the presentation, it’s a way to make sagas, which are inherently ACD, ACID again.
In a later post, I’ll describe how to implement this saga.
Microservices.io is brought to you by Chris Richardson. Experienced software architect, author of POJOs in Action, the creator of the original CloudFoundry.com, and the author of Microservices patterns.
Chris helps clients around the world adopt the microservice architecture through consulting engagements, and training workshops.
Chris teaches comprehensive workshops for architects and developers that will enable your organization use microservices effectively.
Avoid the pitfalls of adopting microservices and learn essential topics, such as service decomposition and design and how to refactor a monolith to microservices.Learn more
Chris offers numerous other resources for learning the microservice architecture.
Want to see an example? Check out Chris Richardson's example applications. See code
Got a specific microservice architecture-related question? For example:
Consider signing up for a two hour, highly focussed, consulting session.
My virtual bootcamp, distributed data patterns in a microservice architecture, is now open for enrollment!
It covers the key distributed data management patterns including Saga, API Composition, and CQRS.
It consists of video lectures, code labs, and a weekly ask-me-anything video conference repeated in multiple timezones.
The regular price is $395/person but use coupon MECNPWNR to sign up for $120 (valid until May 16th, 2023). There are deeper discounts for buying multiple seats.
Take a look at my Manning LiveProject that teaches you how to develop a service template and microservice chassis.
Engage Chris to create a microservices adoption roadmap and help you define your microservice architecture,
Use the Eventuate.io platform to tackle distributed data management challenges in your microservices architecture.
Eventuate is Chris's latest startup. It makes it easy to use the Saga pattern to manage transactions and the CQRS pattern to implement queries.
Engage Chris to conduct an architectural assessment.
Note: tagging is work-in-process
anti-patterns · application api · application architecture · architecting · architecture documentation · assemblage · beer · containers · dark energy and dark matter · deployment · design-time coupling · development · devops · docker · eventuate platform · glossary · hexagonal architecture · implementing commands · implementing queries · inter-service communication · kubernetes · loose coupling · microservice architecture · microservice chassis · microservices adoption · microservicesio updates · multi-architecture docker images · observability · pattern · refactoring to microservices · resilience · sagas · security · service api · service collaboration · service design · service discovery · service granularity · service template · software delivery metrics · success triangle · tacos · team topologies · transaction management · transactional messaging