Microservice Architecture
Supported by KongPattern: Idempotent Consumer
Context
In an enterprise application, it’s usually a good idea to use a message broker that guarantees at-least once delivery.
At-least once delivery guarantees that a message broker will deliver a message to a consumer even if errors occur.
One side-effect, however, is that the consumer can be invoked repeatedly for the same message.
Consequently, a consumer must be idempotent: the outcome of processing the same message repeatedly must be the same as processing the message once.
If a consumer is not idempotent, multiple invocations can cause bugs.
For example, a consumer of an AccountDebited message that subtracts the debit amount from the current balance would calculate the incorrect balance.
Problem
How does a message consumer handle duplicate messages correctly?
Solution
Implement an idempotent consumer, which is a message consumer that can handle duplicate messages correctly. Some consumers are naturally idempotent. Others must track the messages that they have processed in order to detect and discard duplicates.
You can make a consumer idempotent by recording in the database the IDs of the messages that it has processed successfully.
When processing a message, a consumer can detect and discard duplicates by querying the database.
There are a couple of different places to store the message IDs.
One option is for the consumer to use a separate PROCESSED_MESSAGES table.
The other option is for the consumer to store the IDs in the business entities that it creates or updates.
See also
- The Eventuate framework implements this pattern.
- This blog post about this pattern
pattern
About Microservices.io
Microservices.io is brought to you by Chris Richardson. Experienced software architect, author of POJOs in Action, the creator of the original CloudFoundry.com, and the author of Microservices patterns.
Chris helps clients around the world adopt the microservice architecture through consulting engagements, and training workshops.
Remote consulting session
Got a specific microservice architecture-related question? For example:
- Wondering whether your organization should adopt microservices?
- Want to know how to migrate your monolith to microservices?
- Facing a tricky microservice architecture design problem?
Consider signing up for a two hour, highly focussed, consulting session.
Virtual bootcamp: Distributed data patterns in a microservice architecture
My virtual bootcamp, distributed data patterns in a microservice architecture, is now open for enrollment!
It covers the key distributed data management patterns including Saga, API Composition, and CQRS.
It consists of video lectures, code labs, and a weekly ask-me-anything video conference repeated in multiple timezones.
The regular price is $395/person but use coupon KGBJMRAM to sign up for $195 (valid until March 8th, 2023). There are deeper discounts for buying multiple seats.
Learn how to create a service template and microservice chassis
Take a look at my Manning LiveProject that teaches you how to develop a service template and microservice chassis.
LEARN about microservices
Chris offers numerous resources for learning the microservice architecture.
Training classes
Chris teaches comprehensive workshops, training classes and bootcamps for executives, architects and developers to help your organization use microservices effectively.
Avoid the pitfalls of adopting microservices and learn essential topics, such as service decomposition and design and how to refactor a monolith to microservices.
Delivered in-person and remotely.
Get the book: Microservices Patterns
Read Chris Richardson's book:
Example microservices applications
Want to see an example? Check out Chris Richardson's example applications. See code
BUILD microservices
Consulting services
Engage Chris to create a microservices adoption roadmap and help you define your microservice architecture,
The Eventuate platform
Use the Eventuate.io platform to tackle distributed data management challenges in your microservices architecture.
Eventuate is Chris's latest startup. It makes it easy to use the Saga pattern to manage transactions and the CQRS pattern to implement queries.
ASSESS your architecture
Consulting services
Engage Chris to conduct an architectural assessment.
Join the microservices google group
Topics
Note: tagging is work-in-process
anti-patterns · application api · application architecture · architecting · architecture documentation · assemblage · dark energy and dark matter · deployment · design-time coupling · development · devops · docker · glossary · hexagonal architecture · implementing commands · implementing queries · inter-service communication · loose coupling · microservice architecture · microservice chassis · microservices adoption · microservicesio updates · multi-architecture docker images · observability · pattern · refactoring to microservices · resilience · sagas · security · service api · service collaboration · service design · service discovery · service granularity · service template · software delivery metrics · success triangle · team topologies · transaction management · transactional messaging
The patterns
Application architecture patterns
Decomposition
- Decompose by business capability
- Decompose by subdomain
- Self-contained Servicenew
- Service per teamnew
Refactoring to microservicesnew
Data management
Transactional messaging
Testing
Deployment patterns
- Multiple service instances per host
- Service instance per host
- Service instance per VM
- Service instance per Container
- Serverless deployment
- Service deployment platform
Cross cutting concerns
Communication style
External API
Service discovery
- Client-side discovery
- Server-side discovery
- Service registry
- Self registration
- 3rd party registration
Reliability
Security
Observability
- Log aggregation
- Application metrics
- Audit logging
- Distributed tracing
- Exception tracking
- Health check API
- Log deployments and changes
UI patterns