Pattern: Transaction log tailing

pattern   transactional messaging   inter-service communication  

Context

You have applied the Transactional Outbox pattern.

Problem

How to publish messages/events into the outbox in the database to the message broker?

Forces

Solution

Tail the database transaction log and publish each message/event inserted into the outbox to the message broker.

The mechanism for trailing the transaction log depends on the database:

  • MySQL binlog
  • Postgres WAL
  • AWS DynamoDB table streams

Example

Resulting context

This pattern has the following benefits:

  • No 2PC
  • Guaranteed to be accurate

This pattern has the following drawbacks:

  • Relatively obscure although becoming increasing common
  • Requires database specific solutions
  • Tricky to avoid duplicate publishing

Learn more


pattern   transactional messaging   inter-service communication  


Copyright © 2024 Chris Richardson • All rights reserved • Supported by Kong.

About Microservices.io

Microservices.io is brought to you by Chris Richardson. Experienced software architect, author of POJOs in Action, the creator of the original CloudFoundry.com, and the author of Microservices patterns.

ASK CHRIS

?

Got a question about microservices?

Fill in this form. If I can, I'll write a blog post that answers your question.

NEED HELP?

I help organizations improve agility and competitiveness through better software architecture.

Learn more about my consulting engagements, and training workshops.

LEARN about microservices

Chris offers numerous other resources for learning the microservice architecture.

Get the book: Microservices Patterns

Read Chris Richardson's book:

Example microservices applications

Want to see an example? Check out Chris Richardson's example applications. See code

Virtual bootcamp: Distributed data patterns in a microservice architecture

My virtual bootcamp, distributed data patterns in a microservice architecture, is now open for enrollment!

It covers the key distributed data management patterns including Saga, API Composition, and CQRS.

It consists of video lectures, code labs, and a weekly ask-me-anything video conference repeated in multiple timezones.

The regular price is $395/person but use coupon NPXJKULI to sign up for $95 (valid until December 25th, 2024). There are deeper discounts for buying multiple seats.

Learn more

Learn how to create a service template and microservice chassis

Take a look at my Manning LiveProject that teaches you how to develop a service template and microservice chassis.

Signup for the newsletter


BUILD microservices

Ready to start using the microservice architecture?

Consulting services

Engage Chris to create a microservices adoption roadmap and help you define your microservice architecture,


The Eventuate platform

Use the Eventuate.io platform to tackle distributed data management challenges in your microservices architecture.

Eventuate is Chris's latest startup. It makes it easy to use the Saga pattern to manage transactions and the CQRS pattern to implement queries.


Join the microservices google group