A service command typically needs to create/update/delete aggregates in the database and send messages/events to a message broker. For example, a service that participates in a saga needs to update business entities and send messages/events. Similarly, a service that publishes a domain event must update an aggregate and publish an event.
The command must atomically update the database and send messages in order to avoid data inconsistencies and bugs. However, it is not viable to use a traditional distributed transaction (2PC) that spans the database and the message broker The database and/or the message broker might not support 2PC. And even if they do, it’s often undesirable to couple the service to both the database and the message broker.
But without using 2PC, sending a message in the middle of a transaction is not reliable. There’s no guarantee that the transaction will commit. Similarly, if a service sends a message after committing the transaction there’s no guarantee that it won’t crash before sending the message.
In addition, messages must be sent to the message broker in the order they were sent by the service.
They must usually be delivered to each consumer in the same order although that’s outside the scope of this pattern.
For example, let’s suppose that an aggregate is updated by a series of transactions T1
, T2
, etc.
This transactions might be performed by the same service instance or by different service instances.
Each transaction publishes a corresponding event: T1 -> E1
, T2 -> E2
, etc.
Since T1
precedes T2
, event E1
must be published before E2
.
How to atomically update the database and send messages to a message broker?
A good solution to this problem is to use event sourcing. Event sourcing persists the state of a business entity such an Order or a Customer as a sequence of state-changing events. Whenever the state of a business entity changes, a new event is appended to the list of events. Since saving an event is a single operation, it is inherently atomic. The application reconstructs an entity’s current state by replaying the events.
Applications persist events in an event store, which is a database of events. The store has an API for adding and retrieving an entity’s events. The event store also behaves like a message broker. It provides an API that enables services to subscribe to events. When a service saves an event in the event store, it is delivered to all interested subscribers.
Some entities, such as a Customer, can have a large number of events. In order to optimize loading, an application can periodically save a snapshot of an entity’s current state. To reconstruct the current state, the application finds the most recent snapshot and the events that have occurred since that snapshot. As a result, there are fewer events to replay.
Customers and Orders is an example of an application that is built using Event Sourcing and CQRS. The application is written in Java, and uses Spring Boot. It is built using Eventuate, which is an application platform based on event sourcing and CQRS.
The following diagram shows how it persist orders.
Instead of simply storing the current state of each order as a row in an ORDERS
table, the application persists each Order
as a sequence of events.
The CustomerService
can subscribe to the order events and update its own state.
Here is the Order
aggregate:
public class Order extends ReflectiveMutableCommandProcessingAggregate<Order, OrderCommand> {
private OrderState state;
private String customerId;
public OrderState getState() {
return state;
}
public List<Event> process(CreateOrderCommand cmd) {
return EventUtil.events(new OrderCreatedEvent(cmd.getCustomerId(), cmd.getOrderTotal()));
}
public List<Event> process(ApproveOrderCommand cmd) {
return EventUtil.events(new OrderApprovedEvent(customerId));
}
public List<Event> process(RejectOrderCommand cmd) {
return EventUtil.events(new OrderRejectedEvent(customerId));
}
public void apply(OrderCreatedEvent event) {
this.state = OrderState.CREATED;
this.customerId = event.getCustomerId();
}
public void apply(OrderApprovedEvent event) {
this.state = OrderState.APPROVED;
}
public void apply(OrderRejectedEvent event) {
this.state = OrderState.REJECTED;
}
Here is an example of an event handler in the CustomerService
that subscribes to Order
events:
@EventSubscriber(id = "customerWorkflow")
public class CustomerWorkflow {
@EventHandlerMethod
public CompletableFuture<EntityWithIdAndVersion<Customer>> reserveCredit(
EventHandlerContext<OrderCreatedEvent> ctx) {
OrderCreatedEvent event = ctx.getEvent();
Money orderTotal = event.getOrderTotal();
String customerId = event.getCustomerId();
String orderId = ctx.getEntityId();
return ctx.update(Customer.class, customerId, new ReserveCreditCommand(orderTotal, orderId));
}
}
It processes an OrderCreated
event by attempting to reserve credit for the orders customer.
There are several example applications that illustrate how to use event sourcing.
Event sourcing has several benefits:
Event sourcing also has several drawbacks:
Microservices.io is brought to you by Chris Richardson. Experienced software architect, author of POJOs in Action, the creator of the original CloudFoundry.com, and the author of Microservices patterns.
Chris helps clients around the world adopt the microservice architecture through consulting engagements, and training workshops.
Chris teaches comprehensive workshops for architects and developers that will enable your organization use microservices effectively.
Avoid the pitfalls of adopting microservices and learn essential topics, such as service decomposition and design and how to refactor a monolith to microservices.
Learn moreChris offers numerous other resources for learning the microservice architecture.
Want to see an example? Check out Chris Richardson's example applications. See code
Got a specific microservice architecture-related question? For example:
Consider signing up for a two hour, highly focussed, consulting session.
My virtual bootcamp, distributed data patterns in a microservice architecture, is now open for enrollment!
It covers the key distributed data management patterns including Saga, API Composition, and CQRS.
It consists of video lectures, code labs, and a weekly ask-me-anything video conference repeated in multiple timezones.
The regular price is $395/person but use coupon MECNPWNR to sign up for $120 (valid until May 16th, 2023). There are deeper discounts for buying multiple seats.
Take a look at my Manning LiveProject that teaches you how to develop a service template and microservice chassis.
Engage Chris to create a microservices adoption roadmap and help you define your microservice architecture,
Use the Eventuate.io platform to tackle distributed data management challenges in your microservices architecture.
Eventuate is Chris's latest startup. It makes it easy to use the Saga pattern to manage transactions and the CQRS pattern to implement queries.
Engage Chris to conduct an architectural assessment.
Note: tagging is work-in-process
anti-patterns · application api · application architecture · architecting · architecture documentation · assemblage · beer · containers · dark energy and dark matter · deployment · design-time coupling · development · devops · docker · eventuate platform · glossary · hexagonal architecture · implementing commands · implementing queries · inter-service communication · kubernetes · loose coupling · microservice architecture · microservice chassis · microservices adoption · microservicesio updates · multi-architecture docker images · observability · pattern · refactoring to microservices · resilience · sagas · security · service api · service collaboration · service design · service discovery · service granularity · service template · software delivery metrics · success triangle · tacos · team topologies · transaction management · transactional messaging
Application architecture patterns
Decomposition
Refactoring to microservicesnew
Data management
Transactional messaging
Testing
Deployment patterns
Cross cutting concerns
Communication style
External API
Service discovery
Reliability
Security
Observability
UI patterns