Managing data consistency in a microservice architecture using Sagas - Implementing a choreography-based saga

sagas   transaction management  

This is the third in a series of posts that expands on my recent MicroCPH talk on Managing data consistency in a microservice architecture using Sagas (slides, video).

The other posts in this series are:

In part 1, I introduced the concept of a saga and part 2, I described two different saga coordination mechanisms:

  • choreography - the saga participants collaborate by exchanging events
  • orchestration - a centralized orchestrator invokes the saga participants using request/asynchronous responsive

In a later post, I’ll describe how to implement orchestration-based sagas. In this post, I describe the design and implementation of a choreography-based saga.

The choreography-based Create Order saga

In a choreography-based saga, the saga participants collaborate by exchanging events. Each step of a choreography-based saga updates the database (e.g. an aggregate) and publishes a domain event. The first step of a saga is initiated by a command that’s invoked by an external request, such an HTTP POST. Each subsequent step is triggered by an event emitted by a previous step.

Here is the definition of the choreography-based version of the Create Order saga:

Step Triggering event Participant Command Events
1 - Order Service createPendingOrder() OrderCreated
2 OrderCreated Customer Service reserveCredit() Credit Reserved, Credit Limit Exceeded
3a Credit Reserved Order Service approveOrder() -
3b Credit Limit Exceeded Order Service rejectOrder() -

Each step, except of first, has a triggering event. The step of the saga executes a command, which updates its data and emits an event. Since step 2 can emit one of two possible events, the saga can branch, resulting in steps 3a and 3b.

The following diagram shows the flow:

The flow is as follows:

  1. The Order Service receives the POST /orders request and creates an Order in a PENDING state
  2. It then emits an Order Created event
  3. The Customer Service’s event handler attempts to reserve credit
  4. It then emits an event indicating the outcome
  5. The OrderService’s event handler either approves or rejects the Order

Implementing a choreography-based saga

Let’s take a look at how the Order Service and Customer Service implement the choreography-based Create Order saga. This example code is developed using Eventuate Tram, which is a platform that solves the distributed data management problems inherent in a microservice architecture. The complete example code is in the eventuate-tram-examples-customers-and-orders Github repository.

The Order Service

The saga is initiated in the Order Service when the client makes a POST /orders request. The @Controller that handles the request calls OrderService.createOrder().

@Transactional
public class OrderService {

  @Autowired
  private DomainEventPublisher domainEventPublisher;

  @Autowired
  private OrderRepository orderRepository;

  public Order createOrder(OrderDetails orderDetails) {
    ResultWithEvents<Order> orderWithEvents = Order.createOrder(orderDetails);
    Order order = orderWithEvents.result;
    orderRepository.save(order);
    domainEventPublisher.publish(Order.class, order.getId(), orderWithEvents.events);
    return order;
  }

The createOrder() method, which is executed within a Spring-managed transaction persists the newly created Order in the database using Spring Data for JPA and publishes an OrderCreated event using the DomainEventPublisher.

The DomainEventPublisher class is provided by the Eventuate Tram framework. It implements the Transactional Outbox pattern and inserts serialized events into a MESSAGE table. The Eventuate CDC service then publishes those events to the message broker using either transaction log tailing or polling.

Here is one of the @Configuration classes for the Order Service:

@Configuration
@EnableJpaRepositories
@EnableAutoConfiguration
@Import({TramJdbcKafkaConfiguration.class,
        TramEventsPublisherConfiguration.class,
        TramEventSubscriberConfiguration.class})
public class OrderConfiguration {

  @Bean
  public OrderService orderService(DomainEventPublisher domainEventPublisher,   
                                   OrderRepository orderRepository) {
    return new OrderService(domainEventPublisher, orderRepository);
  }

This @Configuration class @Imports the required Eventuate Tram @Configuration classes and enables Spring Data for JPA repositories. It also defines the orderService @Bean.

The Order Service also has event handlers for Customer Service events. But rather than look at those, let’s look at the event handlers for the Customer Service, which are more interesting.

The Customer Service

The Customer Service subscribes to events published by the Order Service. It has event handlers for Order events including Order Created and Order Cancelled. The following listing shows the event handler for the Order Created event:

public class OrderEventConsumer {
  private Logger logger = LoggerFactory.getLogger(getClass());

  @Autowired
  private CustomerRepository customerRepository;

  @Autowired
  private DomainEventPublisher domainEventPublisher;

  public DomainEventHandlers domainEventHandlers() {
    return DomainEventHandlersBuilder
            .forAggregateType("io.eventuate.examples.tram.ordersandcustomers.orders.domain.Order")
            .onEvent(OrderCreatedEvent.class, this::handleOrderCreatedEventHandler)
            .onEvent(OrderCancelledEvent.class, this::handleOrderCancelledEvent)
            .build();
  }

  public void handleOrderCreatedEventHandler(
             DomainEventEnvelope<OrderCreatedEvent> domainEventEnvelope) {

    Long orderId = Long.parseLong(domainEventEnvelope.getAggregateId());

    OrderCreatedEvent orderCreatedEvent = domainEventEnvelope.getEvent();

    Long customerId = orderCreatedEvent.getOrderDetails().getCustomerId();

    Optional<Customer> possibleCustomer = customerRepository.findById(customerId);

    if (!possibleCustomer.isPresent()) {
      logger.info("Non-existent customer: {}", customerId);
      domainEventPublisher.publish(Customer.class,
              customerId,
              Collections.singletonList(new CustomerValidationFailedEvent(orderId)));
      return;
    }

    Customer customer = possibleCustomer.get();


    try {
      customer.reserveCredit(orderId, orderCreatedEvent.getOrderDetails().getOrderTotal());

      CustomerCreditReservedEvent customerCreditReservedEvent =
              new CustomerCreditReservedEvent(orderId);

      domainEventPublisher.publish(Customer.class,
              customer.getId(),
              Collections.singletonList(customerCreditReservedEvent));

    } catch (CustomerCreditLimitExceededException e) {

      CustomerCreditReservationFailedEvent customerCreditReservationFailedEvent =
              new CustomerCreditReservationFailedEvent(orderId);

      domainEventPublisher.publish(Customer.class,
              customer.getId(),
              Collections.singletonList(customerCreditReservationFailedEvent));
    }
  }

The event handler:

  1. Attempts to retrieve the Customer from the database and reserve credit
  2. If the Customer is not found, it publishes a CustomerValidationFailedEvent
  3. If credit is successfully reserved, it publishes a CustomerCreditReservedEvent
  4. Otherwise, if the credit limit is exceeded it publishes a CustomerCreditReservationFailedEvent

Here is the Spring @Configuration class that configures the event handlers.

@Configuration
public class CustomerConfiguration {

  @Bean
  public OrderEventConsumer orderEventConsumer() {
    return new OrderEventConsumer();
  }

  @Bean
  public DomainEventDispatcher   
     domainEventDispatcher(OrderEventConsumer orderEventConsumer,
                           DomainEventDispatcherFactory domainEventDispatcherFactory) {
    return domainEventDispatcherFactory.make(
       "orderServiceEvents", orderEventConsumer.domainEventHandlers());
  }
  ...

This @Configuration class’s primary responsibility is to subscribe to the Order domain events. It does that using the DomainEventDispatcherFactory, which is provided by Eventuate Tram.

As you have just seen, the Create Order saga is implemented by a simple exchange of events between the Order Service and the Customer Service. However, while this example is quite straightforward, choreography-based sagas have several drawbacks. For example, the implementation of the saga is scattered around the participating services, which can make more complex sagas difficult to understand. Please see my book for a more detailed explanation of the choreography-based sagas.

In a later blog post, I’ll describe how to implement the Create Order saga using orchestration.

To learn more


sagas   transaction management  


Copyright © 2024 Chris Richardson • All rights reserved • Supported by Kong.

About Microservices.io

Microservices.io is brought to you by Chris Richardson. Experienced software architect, author of POJOs in Action, the creator of the original CloudFoundry.com, and the author of Microservices patterns.

NEED HELP?

I help organizations improve agility and competitiveness through better software architecture.

Learn more about my consulting engagements, and training workshops.

PREMIUM CONTENT

Premium content and office hours is now available for paid subscribers at premium.microservices.io.

MICROSERVICES WORKSHOPS

Chris teaches comprehensive workshops for architects and developers that will enable your organization use microservices effectively.

Avoid the pitfalls of adopting microservices and learn essential topics, such as service decomposition and design and how to refactor a monolith to microservices.

Learn more

LEARN about microservices

Chris offers numerous other resources for learning the microservice architecture.

Get the book: Microservices Patterns

Read Chris Richardson's book:

Example microservices applications

Want to see an example? Check out Chris Richardson's example applications. See code

Remote consulting session

Got a specific microservice architecture-related question? For example:

  • Wondering whether your organization should adopt microservices?
  • Want to know how to migrate your monolith to microservices?
  • Facing a tricky microservice architecture design problem?

Consider signing up for a two hour, highly focussed, consulting session.

Virtual bootcamp: Distributed data patterns in a microservice architecture

My virtual bootcamp, distributed data patterns in a microservice architecture, is now open for enrollment!

It covers the key distributed data management patterns including Saga, API Composition, and CQRS.

It consists of video lectures, code labs, and a weekly ask-me-anything video conference repeated in multiple timezones.

The regular price is $395/person but use coupon ILFJODYS to sign up for $95 (valid until April 12, 2024). There are deeper discounts for buying multiple seats.

Learn more

Learn how to create a service template and microservice chassis

Take a look at my Manning LiveProject that teaches you how to develop a service template and microservice chassis.

Signup for the newsletter


BUILD microservices

Ready to start using the microservice architecture?

Consulting services

Engage Chris to create a microservices adoption roadmap and help you define your microservice architecture,


The Eventuate platform

Use the Eventuate.io platform to tackle distributed data management challenges in your microservices architecture.

Eventuate is Chris's latest startup. It makes it easy to use the Saga pattern to manage transactions and the CQRS pattern to implement queries.

ASSESS your architecture

Assess your application's microservice architecture and identify what needs to be improved.

Consulting services

Engage Chris to conduct an architectural assessment.



Join the microservices google group