Designing an architecture using dark matter and dark energy

Updated March 2023

When designing an application’s architecture there are numerous decisions that you must make. For example:

1. Should the application have a monolithic or a microservice architecture?
2. If you use the microservice architecture, what are the best service boundaries?
3. How to design an operation that spans multiple services?

When working with clients, I’m often asked “What’s the best way to do X?”. Almost always, my answer is “It depends!”. Specifically, the best solution depends on the specific context within which you are trying to solve the problem. Sadly, there’s no silver bullet. Instead, I recommend using criteria based on dark matter and dark energy to drive the design of the architecture.

Dark energy and dark matter drive architecture design

Dark matter and dark energy are mysterious concepts invented by astrophysicists in order to explain certain astronomical observations. Dark energy is an anti-gravity that is forcing matter apart and accelerating the expansion of the universe. Dark matter is an invisible matter that has a gravitational effect on stars and galaxies. As I described in my iSAQB and Mucon presentations, I use them as metaphors for the repulsive and attractive forces that must be resolved when designing an architecture.

But Before talking about how to apply the metaphors of dark matter and dark energy, I want to first talk about two inter-related concepts: system operations and subdomains.

About system operations and subdomains

An application’s system operations and subdomains are an implementable model of its functionality.

System operations

System operations are a distillation of an application’s functional requirements (e.g. stories). Each one is an abstract representation of a request that an application must handle. For example, the FTGO application implements system operations, such as createOrder(), cancelOrder(), and findOrderHistory(). System operations also model actions that are triggered by the passing of time.

The first step of the Assemblage architecture definition process distills the application’s functional requirements into system operations. These system operations play a central role is the rest of the process.

System subdomains

An application’s functionality as described by the systems operations is implemented by a collection of subdomains. A subdomain is an implementable, (usually) object-oriented model of a business capability. It’s owned by a small team, which is responsible for its development. For example, the FTGO application has numerous subdomains including Order Management, Delivery Management, Restaurant Management, Kitchen Management, etc. And, since it’s a Java application, each subdomain is a collection of Java packages that contain classes.

Each system operation spans one or more subdomains.
For example, the createOrder() system operation spans numerous domains including Order Management, Restaurant Management, Kitchen Management, and Accounting.

The second step of the Assemblage architecture definition process designs the application’s subdomains.

An application’s subdomains comprise the logical (a.k.a. development) view of its architecture. We also need to consider the implementation view, which defines how those subdomains are packaged as components, which are executable/deployable units.

What’s the architecture’s implementation view

The implementation view defines the application’s modules and components. A module is a compiled and packaged subdomain. A component is an executable or deployable and consists of one or more modules. In a Java application, a module is a JAR file and so each subdomain would be packaged as a one or more JAR files. A Java component is a WAR file or an executable fat JAR file. that contains a collection of JAR files.

If an application consists of a single component then it has a monolithic architecture. Alternatively, if it consists of two or more components then it has a microservice architecture.

Defining an application’s implementation view

Given the above definitions of subdomains and components, the problem of defining (the implementation view of) an architecture becomes

1. How to partition an application’s subdomains into one or more components, a.k.a. services? For example, given three subdomains: A, B, C, there are numerous ways to partition them into three services: ABC; AB + C; A + B + C; AB + C; AC + B; A + BC.
2. Given a partitioning, how to design those system operations that span services?

It’s imporatnt to keep in mind that these two problems are inter-related. For example, a given partitioning might prevent you from implementing a system operation efficiently. As a result, you might need to redesign the partitioning.

Let’s now look at how the metaphors of dark energy and dark matter can help you solve these problems.

About dark energy and dark matter

The following diagram shows the dark energy and dark matter forces. There are ten forces in total. Five repulsive dark energy forces and five attractive dark matter forces.

Let’s first take a look at dark energy.

Using dark energy

Astrophysicists invented the concept of dark energy to explain why the expansion of the universe, which was previously slowing down, started to accelerate 7 billion years ago. The theory is that as the universe expanded gravity weakened and eventually dark energy, which is an anti-gravity repulsive force, became dominant and accelerated the expansion. Dark energy a useful metaphor to explain the repulsive forces that encourage the partitioning of subdomains into more than one service.

There are five dark energy forces that act on subdomains and encourage the partitioning of subdomains into multiple services:

• Simple components - simple components are easier to understand and maintain than complex components
• Team autonomy - a team needs to be able to develop, test and deploy their software independently
• Fast deployment pipeline - fast feedback and high deployment frequency are essential and are enabled by a fast deployment pipeline
• Support multiple technology stacks - subdomains are sometimes implemented using a variety of technologies; and developers need to evolve the application’s technology stack, e.g. use current versions of languages and frameworks
• Segregate by characteristics - subdomains can be segregated by their characteristics such as their resource requirements, their availability requirements, their security requirements, etc.

These repulsive forces act on subdomains and encourage them to be packaged as separate services. However, they aren’t the only forces you must consider. You must also consider attractive forces.

Using dark matter

When astrophysicists looked at orbits of stars in distant galaxies, they discovered that they moved faster than expected based on the amount of visible matter. To explain this discrepancy, they invented the concept of dark matter, which is an invisible form of matter that exerts a gravitational attraction. It’s a useful metaphor for the attractive forces in a microservice architecture that resist decomposition into services. These attractive forces that act on subdomains are generated by system operations that span those subdomains.

There are five dark matter attractive forces that resist decomposition:

• Simple interactions - simple interactions between components are easier to understand and troubleshoot than complex interactions
• Efficient interactions - lots of network round trips and large data transfers can be too inefficient
• Prefer ACID over BASE - it’s easier to implement an operation as an ACID transaction rather than, for example, eventually consistent sagas
• Minimize runtime coupling - in order to maximize availability and reduce latency
• Minimize design time coupling - reduce the likelihood of changing services in lockstep, which reduces productivity

The strength of the attractive forces that an particular system operation generates depends on two factors:

1. How the subdomains that it references are partitioned into services
2. The patterns that used to implement the operation

Sometimes, you can reduce the strength of those forces by redesigning the operation. Alternatively, you might need to repartition the subdomains.

Designing an architecture = balancing dark matter and dark energy forces

One reason the monolithic architecture is so appealing is that it satisfies all of attractive forces. Sadly, however, it neglects all of the repulsive forces especially when the application is large. At the other end of the spectrum, a fine-grained microservice architecture satisfies all of the repulsive forces yet might neglect the attractive forces. It might be a fragile distributed monolith that combines the worst of both architectural styles due to excessive design-time and runtime coupling.

The job of architect is to design an architecture that minimizes the repulsive forces within components AND minimizes the attractive forces between components. Future blog posts will do a deeper dive into dark energy and dark matter forces.

To learn more about designing a microservice architecture

Take a look at Introducing Assemblage - a microservice architecture definition process. Assemblage uses the concepts of dark energy and dark matter to design a microservice architecture.

To learn more about dark energy, dark matter and microservices

See these presentations:

• Mucon 2021 - Dark energy, dark matter: imperfect metaphors for designing microservices
• iSAQB keynote - Modular monoliths and microservices: architectural patterns for rapid, reliable, frequent and sustainable development
• Dark energy, dark matter and microservice architecture collaboration patterns
• QCONSF 2022: Dark Energy, Dark Matter and the Microservices Patterns?!
• YOW London 2022: Considering Migrating a Monolith to Microservices? A Dark Energy, Dark Matter Perspective

Need help with microservices?

I provide consulting and training to help you use the microservice architecture effectively.