sagas   transaction management  

Managing data consistency in a microservice architecture using Sagas part 2 - coordinating sagas

This is the second in a series of posts that expands on my recent MicroCPH talk on Managing data consistency in a microservice architecture using Sagas (slides, video).

The other posts in this series are:

In part 1, I introduced the concept of a saga. I also described the Customers and Orders example, which uses a saga to create an Order. In this post I’m going to describe how to coordinate sagas.

Coordinating sagas

As I described in part 1, a saga consists of a sequence of steps. Each step is a local transaction in a participating service. For example, the happy path for the Create Order saga is as follows:

  1. Order Service : createPendingOrder()
  2. Customer Service : reserveCredit()
  3. Order Service : approveOrder()

Similarly, the flow when there is insufficient credit is as follows:

  1. Order Service : createPendingOrder()
  2. Customer Service : reserveCredit()
  3. Order Service : rejectOrder()

The services must implement a coordination mechanism that decides the sequencing of the steps and triggers their execution.

Choreography vs. Orchestration

There are two main ways to coordinate sagas: choreography and orchestration. The two approaches differ in the nature of the asynchronous communication and whether coordination is centralized or distributed.

Choreography is an event-driven approach. When using choreography, there isn’t a central coordinator telling the saga participants what to do. Instead, the saga participants subscribe to each other’s events and respond accordingly. When a service updates its data, it simply emits a domain event announcing what it has done. Other services subscribe to those events, which trigger updates that emit additional events. For example, in the Create Order saga, the Order Service emits an OrderCreated event, which causes the Customer Service to attempt to reserve credit. In a later blog post, I’ll describe how to implement the Create Order saga using choreography in more detail.

Orchestration uses a saga orchestrator that tells the saga participants what to do. The saga orchestrator communicates with the participants using request/asynchronous response-style interaction. To execute a saga step, it sends a command message to a participant telling it what operation to perform. After the saga participant has performed the operation, it sends a reply message to the orchestrator. The orchestrator then processes the reply message and determines which saga step to perform next. For example, the Create Order saga orchestrator sends a Reserve Credit command to the Customer Service, which then attempts to reserve credit. In a later blog post, I’ll describe how to implement the Create Order saga using orchestration in more detail.

Saga participants communicate asynchronously using a message broker

As you have just seen, saga participants communicate using asynchronous messaging: either events or asynchronous request/response.

In order for the communication to be reliable, it’s essential that the saga participants use a message broker that guarantees at-least-once delivery and has durable subscriptions. That’s because at-least-once delivery and durable subscriptions ensure that a saga completes even if a participant is temporarily unavailable. A message will sit in the message broker’s channel (e.g. queue or topic) until the participant is able to successfully process it.

How saga participants atomically update the database and send a message

Each step of saga updates a database (e.g. a business object or saga orchestrator) and sends a message/event. These two actions must be done atomically in order to avoid data inconsistencies and bugs. For example, if a service sent a message after committing the database transaction there is a risk of it crashing before sending. Similarly, if a service sends a message in the middle of a database transaction there is a risk that the transaction will be rolled back. In both scenarios, the application is left in an inconsistent state.

In order to avoid this problem, a saga must use one of the following patterns:

  • Transactional Outbox pattern - publish a message by inserting it into an OUTBOX table as part of the database transaction. A separate process (e.g. the Eventuate CDC) retrieves the message from the OUTBOX table and sends it to the message broker.
  • Event Sourcing pattern - uses events to persist domain objects and thereby combines updating the database and publishing events into a single, inherently atomic operation.

In later blog posts, I’ll describe how to implement the Create Order saga using choreography and orchestration.

To learn more


sagas   transaction management  


Copyright © 2023 Chris Richardson • All rights reserved • Supported by Kong.

About Microservices.io

Microservices.io is brought to you by Chris Richardson. Experienced software architect, author of POJOs in Action, the creator of the original CloudFoundry.com, and the author of Microservices patterns.

Chris helps clients around the world adopt the microservice architecture through consulting engagements, and training workshops.

Learn how to create a service template and microservice chassis

Take a look at my Manning LiveProject that teaches you how to develop a service template and microservice chassis.

New virtual bootcamp: Distributed data patterns in a microservice architecture

My virtual bootcamp, distributed data patterns in a microservice architecture, is now open for enrollment!

It covers the key distributed data management patterns including Saga, API Composition, and CQRS.

It consists of video lectures, code labs, and a weekly ask-me-anything video conference repeated in multiple timezones.

The regular price is $395/person but use coupon JUNVCEJE to sign up for $195 (valid until February 1st, 2023). There are deeper discounts for buying multiple seats.

Learn more

Signup for the newsletter


LEARN about microservices

Chris offers numerous resources for learning the microservice architecture.

Training classes

Chris teaches comprehensive workshops, training classes and bootcamps for executives, architects and developers to help your organization use microservices effectively.

Avoid the pitfalls of adopting microservices and learn essential topics, such as service decomposition and design and how to refactor a monolith to microservices.

Delivered in-person and remotely.


Get the book: Microservices Patterns

Read Chris Richardson's book:

Example microservices applications

Want to see an example? Check out Chris Richardson's example applications. See code

BUILD microservices

Ready to start using the microservice architecture?

Consulting services

Engage Chris to create a microservices adoption roadmap and help you define your microservice architecture,


The Eventuate platform

Use the Eventuate.io platform to tackle distributed data management challenges in your microservices architecture.

Eventuate is Chris's latest startup. It makes it easy to use the Saga pattern to manage transactions and the CQRS pattern to implement queries.

ASSESS your architecture

Assess your application's microservice architecture and identify what needs to be improved.

Consulting services

Engage Chris to conduct an architectural assessment.



Join the microservices google group

Topics

Note: tagging is work-in-process

anti-patterns   ·  application api   ·  application architecture   ·  architecting   ·  architecture documentation   ·  dark energy and dark matter   ·  deployment   ·  development   ·  devops   ·  docker   ·  implementing commands   ·  implementing queries   ·  inter-service communication   ·  loose coupling   ·  microservice architecture   ·  microservice chassis   ·  microservices adoption   ·  microservicesio updates   ·  multi-architecture docker images   ·  observability   ·  pattern   ·  refactoring to microservices   ·  resilience   ·  sagas   ·  security   ·  service api   ·  service collaboration   ·  service design   ·  service discovery   ·  service granularity   ·  service template   ·  software delivery metrics   ·  success triangle   ·  team topologies   ·  transaction management   ·  transactional messaging

All content


Posts

24 Jul 2017 » Revised data patterns