Microservice architecture

Architecting for uncertainty - part 3: GenAI as an amplifier

Wed, 07 Oct 2026 08:03:00 +0000

This is the third in a series of articles based on my talk Architecting for Fast Flow: Thriving Amid Uncertainty. In this series, I define the term fast flow and explain why it has become an essential software delivery capability. I discuss how fast flow emerges from a socio-technical architecture, which consists of a software architecture and its development organization. Finally, I describe how to design and evolve a socio-technical architecture in order to sustain fast flow.

The articles in the series are:

Part 2 explained that an organization that wants to thrive in a VUCA world must practice fast flow: deliver a continuous stream of small changes to production, and continually learn from feedback from users and from production. Given all the hype, you might think that fast flow can be achieved by simply using GenAI coding agents. This article explains why GenAI alone cannot guarantee fast flow, and how it amplifies an organization’s existing software delivery capabilities, for better or for worse.

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →

Architecting with LLMs: use plain old code when you can, LLMs when you must

Tue, 29 Sep 2026 08:03:00 +0000

I recently wrote about the Coding agent sandwich, which is an architecture for a coding agent harness. The architecture consists of a tasty filling of LLM invocations sandwiched between two slices of plain old code (POC). Its practical rule is: use POC when you can, LLMs when you must.

The following diagram shows how that article’s worked example, the implement plan workflow, applies this rule. The workflow is decomposed into a tree of responsibilities. Each responsibility is implemented using either POC (gray) or an LLM (yellow). LLMs are used only for the responsibilities that require judgment, such as writing code and fixing a broken build. The workflow has orchestration at two levels. POC orchestrates the overall workflow, and the LLM agent that implements each task decides which action to take next within that task.

In this article, I apply that rule and the other ideas from the Coding agent sandwich to architecting systems that leverage LLMs. Let’s start by looking at the timeless idea of the right tool for the right job.

The right tool for the right job

As with many new technologies, GenAI agents have attracted tremendous hype. This naturally leads to the Golden Hammer anti-pattern: the tendency to apply a shiny tool to every problem, regardless of whether it’s the best fit. As the hype settles, it becomes clearer where agents fit and where POC is the better choice.

When to use POC

POC is the right choice when you can specify how to compute the answer. This is the case when the rules are known and correctness can be precisely defined. Don’t waste your time trying to solve a problem with an LLM when a far simpler solution exists.

Beyond being the right fit for these problems, POC has the benefit of repeatability. Given the same inputs and state, POC behaves the same way every time, while an LLM is probabilistic. Repeatability also makes POC easier to develop and debug because failures are reproducible. There’s no need to experiment with different prompts and hope that the LLM does the right thing. And at runtime, POC is typically much faster and cheaper than an LLM invocation.

When to use an LLM

An LLM is a good candidate when you can specify what a good answer looks like but cannot reasonably specify how to compute it. This is the case when the input is ambiguous or unstructured, when understanding natural language is required, or when producing the answer requires judgment or interpretation. For such problems, enumerating deterministic rules is impractical.

Here are some real-world examples where an LLM is a good candidate:

Interpreting an ambiguous operational support request
Translating a user’s business intent into an enterprise transaction
Interpreting complex contractual or engineering requirements
Implementing a software change from a natural-language specification
Diagnosing an unfamiliar production failure

These capabilities come at a price. An LLM invocation is slow and expensive compared to executing POC. And because an LLM is probabilistic, repeated invocations with the same input can behave differently. So before you commit to an LLM, test it against representative examples to check that its error rate, cost, and latency are acceptable.

Start with functional decomposition

Designing a system that uses LLMs starts with recursive functional decomposition. You identify the responsibilities needed to solve the problem. These responsibilities include orchestration, which is the control flow that decides which responsibility to execute next.

The decomposition is hierarchical. A responsibility X can itself be decomposed into sub-responsibilities X1 and X2, together with a coordinator that orchestrates them.

For example, consider the implement plan workflow that is the worked example in the Coding agent sandwich article. Its job is to turn each task in a development plan into a Git commit. Decomposing this problem identifies seven responsibilities:

Getting the next task from the plan
Implementing the task
Validating the result
Pushing the commit and creating a pull request
Waiting for the CI build
Fixing the CI build if it fails
Addressing pull request review feedback

Implementing the task is itself decomposed into writing a test, writing the code, running the tests, marking the task as complete, and committing the changes. The orchestration responsibility is the loop that executes these steps for each task until no tasks remain.

For each responsibility: POC or LLM?

The next step is to decide, for each responsibility, whether to implement it using POC or an LLM. As described earlier, use POC when you can specify how to compute the answer, and use an LLM when you can only specify what a good answer looks like.

When assigning a responsibility to an LLM, keep the responsibility narrowly defined. For example, the implement plan workflow invokes a coding agent for each task in the plan, instead of using one agent run to implement the whole plan. An LLM invocation or agent run that handles many responsibilities is less reliable. The LLM might, for example, stop partway through. A large invocation or run is also wasteful because it uses an expensive LLM for work that POC could do far more cheaply.

For each LLM responsibility: which model?

Once you have decided to implement a responsibility using an LLM, you still need to pick which LLM. LLMs differ in capability, speed, and cost. The more capable models also let you adjust how much reasoning they do per invocation, which affects both cost and latency. A responsibility that requires deep reasoning, such as diagnosing an unfamiliar production failure, needs a large, capable model. A responsibility that requires little reasoning, such as summarizing a support request, can often use a smaller model that is faster and cheaper. Because each responsibility is a separate LLM invocation, you can pick the model for each one. As with the POC-or-LLM decision, test the candidate models against representative examples and compare their error rate, cost, and latency.

Alternative machine learning models

An LLM is just one kind of machine learning model. Use the kind of model whose capability matches the responsibility:

Interpreting natural language - an LLM
Finding similar items - an embedding model
Predicting whether a transaction is fraudulent - a classifier
Predicting tomorrow’s transaction volume - a time-series model
Identifying damage in a photograph - a vision model

A specialized model is typically cheaper, faster, and more accurate at its task than an LLM. The rule therefore generalizes: decompose the system into responsibilities, and implement each one using POC or the machine learning model whose capability matches it.

Orchestration: POC or LLM?

When a system contains an LLM, it’s tempting to assume that the LLM should orchestrate it. But orchestration is itself a responsibility, so the same POC-or-LLM decision applies to it. There are two options:

POC orchestration - the right choice when you can define the steps and the branches between them in advance. The POC orchestrator decides which responsibility to execute next. When a branch requires judgment, the orchestrator can invoke an LLM to choose one of the branches that it defines, and then execute it. Making a decision is simply another narrowly defined responsibility that is implemented using an LLM.
LLM-based orchestration - the right choice when you cannot define the steps in advance. The LLM examines the current state and the results so far, and decides what to invoke next.

The implement plan workflow uses POC orchestration, which forms the top slice of the Coding agent sandwich.

An agent orchestrates and does the work

When an LLM decides each next action after seeing the result of the previous one, the responsibility is implemented as an agent. An agent consists of an LLM, a harness, and a set of tools. The LLM also decides when it has finished. The harness, such as Claude Code, is POC that runs the loop. It invokes the LLM, executes each tool call that the LLM requests, and passes the result back to the LLM.

The LLM in an agent does more than orchestrate. It also performs some of the responsibilities that it orchestrates and delegates the rest to tools. For example, the coding agent that implements a task writes the tests and the code itself, and it runs the tests using Gradle. The coding agent forms the filling of the sandwich.

Give LLMs POC tools

A responsibility implemented using an LLM can itself be decomposed. When it needs to act on the world, such as running tests, querying a database, or invoking an API, each action is a sub-responsibility whose rules are known. Implement each one using POC as a tool with well-defined behavior that the LLM invokes. For lessons on designing such tools, see Migrating a Claude Code skill from WebFetch to the new CircleCI CLI.

For example, the coding agent that implements a task invokes Gradle to run the tests and Git to commit the changes. The LLM decides what to do next based on the actual test results. These tools form the bottom slice of the sandwich.

LLMs often belong at the boundaries

Applying the POC-or-LLM rule to each responsibility tends to put LLMs at the boundaries of a system. At a boundary, the system receives unstructured, ambiguous input from the real world, which must be interpreted. Once the input has been interpreted, it becomes structured state that well-defined operations can act on. Those operations are implemented using POC.

For example, consider a system that handles operational support requests. Each request is written in natural language and is often ambiguous. An LLM interprets the request and converts it into a structured ticket, which specifies the category, the priority, and the affected service. A well-formed ticket can still be wrong, because the LLM might misinterpret the request. POC therefore validates the ticket, for example by checking that the affected service exists, and sends a ticket that fails validation to a person for review. POC then routes each valid ticket and updates the ticketing system. Validation can catch a ticket that names a service that doesn’t exist, but it can’t catch one that names the wrong service.

Not every LLM responsibility is at a boundary. Each LLM responsibility in the implement plan workflow interprets some input, such as a task description, the logs of a failed build, or review comments. But most of its work is writing code, which requires judgment.

Revisiting the POC-or-LLM decision

The decision to implement a responsibility using POC or an LLM can change after the initial implementation. When you first implement a responsibility, you might not understand the problem well enough to specify how to compute the answer, so you use an LLM. As your understanding improves, you might find that you can specify the computation after all. Reimplementing the responsibility using POC then makes it faster, cheaper, and repeatable.

The change can also go the other way. A responsibility that you implemented using POC might turn out to require judgment. For example, its input might be more varied than you expected, so you keep adding rules to handle new cases. In that case, reimplementing the responsibility using an LLM might be simpler than adding more rules.

In either direction, compare the two implementations before switching. Run both against the same representative inputs, and check their results, cost, and latency against the same acceptance criteria.

Summary

The key idea of this article is to design a system that uses LLMs by applying functional decomposition. You decompose the problem into responsibilities, including orchestration, and then decide how to implement each one. To avoid the Golden Hammer anti-pattern, make that decision separately for each responsibility. Use POC whenever you can specify how to compute the answer. Use a machine learning model only when you must, because you can only specify what a good answer looks like. Use an LLM when the responsibility requires interpreting natural language or exercising judgment, and a specialized model, such as a classifier, when it doesn’t.

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →

Architecting for uncertainty - part 2: fast flow defined

Thu, 03 Sep 2026 08:03:00 +0000

This is the second in a series of articles based on my talk Architecting for Fast Flow: Thriving Amid Uncertainty. In this series, I define the term fast flow and explain why it has become an essential software delivery capability. I discuss how fast flow emerges from a socio-technical architecture, which consists of a software architecture and its development organization. Finally, I describe how to design and evolve a socio-technical architecture in order to sustain fast flow.

The articles in the series are:

The first article explained that thriving in a VUCA world requires a business to constantly experiment, and that the more frequently it deploys changes, the more rapidly it learns. This article describes the software delivery capability that makes that possible: fast flow.

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →

Microservices and GenAI in 2026: my Dear Architects conversation

Wed, 02 Sep 2026 08:03:00 +0000

Earlier this year I was a guest on Luca Mezzalira’s Dear Architects podcast. We talked about where microservices are in 2026 and what GenAI changes. My take is that the best software development practices we had before GenAI - you could even say extreme programming - are even more necessary now, because of the nature of GenAI.

Here are the main points. The full conversation is on the episode:

We are still building big balls of mud

Painting with a very broad brush, I think far too many enterprises just muddle along and mostly build big balls of mud.

Software serves two constituencies

Software is an unusual kind of artifact. With most things, how you build one today does not affect how you build the next one. With software, the design decisions you make today affect how easy it is to make changes tomorrow.

Software also serves two constituencies. The end users need features. The organization that builds the software needs it to be easy to change, so that it can keep delivering those features. The second need is the one that gets neglected. The emphasis is on feature development to the exclusion of proper architecture and improving the -ilities. So the software gradually becomes harder to change, and shipping features slows down. As the requirements grow, an application’s architecture has to be improved so that it stays easy to change.

Good practices get devoured

The other part of the problem is what organizations do to good practices. Agile practices came along and got devoured by the organization’s change-resistant antibodies, and morphed into that abomination called SAFe. Then microservices came along and got devoured as well.

The red flags of a distributed monolith

When I start working with an organization, there are a few red flags I look for:

The ratio of services to developers - For a long time my rule of thumb was one service per team. These days I would say you should only have a service if it solves a problem. A really common anti-pattern is to have almost as many services as developers, or more. I repeatedly see a team of five people with five services, asking how to solve some distributed problem across them. Usually the answer is to merge them together, and then you do not have a distributed problem anymore.
A go-faster architecture that is not delivering any faster - Look at your deployment frequency and your lead time.
Testing and releasing services in lockstep - 99% of the reason you have services is so that you can deploy them independently, but then the old QA practice of a system release takes over: we have to test them all together and release them in lockstep.

Either way, you are incurring the cost and complexity of microservices without any of the benefit. They are all symptoms of a distributed monolith.

Dark energy and dark matter

Your default architecture should be a monolith unless you have a clear, tangible problem that introducing one or more services would actually solve. So the question is what problems a service would solve, and conversely, what problems it would create.

I am a science-fiction nerd who is also into software patterns, and the intersection of the two prompted me to come up with the dark energy and dark matter forces. They are the criteria for deciding whether two subdomains should go together in the same service or be separated into different services. I made it up because it sounded really cool, but it turned out to be useful. Both terms come from astrophysics:

Dark energy - the concept astrophysicists use to explain the accelerating expansion of the universe, an anti-gravity that pushes matter apart. It is the metaphor for the forces that push your subdomains apart into separate services. One is the need for a fast deployment pipeline: the more components you have, the smaller each one is, and the faster its deployment pipeline runs. Another is team autonomy: more services means teams can be more autonomous.
Dark matter - the invisible matter that makes up most of the universe. You cannot see it, only its gravitational effect. It is the metaphor for the forces that resist decomposition and pull your subdomains together into a monolith. One is minimize runtime coupling: inter-service communication involves latency and can impact availability. Another is prefer ACID over BASE: operations that span multiple services are eventually consistent, not ACID.

There are five forces pushing your subdomains apart and five pulling them together. I described a couple of each on the episode; you can read about all ten on microservices.io. They are in conflict, and the job of the architect - whether choosing between a monolith and microservices, or defining service boundaries - is to find the set of trade-offs that resolves them.

Context matters

The strength of each force depends on your context: the complexity of your application and the number of teams you have. Team autonomy is an excellent thing, but if you only have one team it is not a reason to have multiple services. Similarly, a small application with a few developers committing regularly has a lightly used deployment pipeline that gives fast feedback. As the application grows and the number of teams grows, each build takes longer and more commits flow into the pipeline, and eventually it becomes a bottleneck. That is when you want to go from a monolith with a single deployment pipeline to multiple services, each with its own.

Low latency example

Luca raised the case of a latency-sensitive system where services are ruled out because of the overhead. Suppose an application has many REST endpoints and only a handful of them are latency sensitive. You put those operations, and all the code they touch, in the same service so that they are not distributed, and the operations that are not latency sensitive can be distributed. Low latency might constrain your service boundaries in certain ways, but it does not necessarily force you to use a monolith.

The microservice architecture is unforgiving

Where you put things matters more in a microservice architecture. In a monolith you can write a class and put it anywhere; it does not matter immediately, though modularity and coupling catch up with you later. In a microservice architecture, where you put things has an immediate impact on latency and coupling, and that is how you can very easily end up with a distributed monolith.

There is more to microservices than services

Delivering software with fast flow takes more than the right architecture. You also need the right organization and the right process. Specifically, you need an organization structured according to Team Topologies, DevOps as defined by the DevOps Handbook, and an architecture that enables both. That is what I call the success triangle. And that is the role of microservices when you are building at scale: a large organization and a large application.

Luca observed that companies often adopt microservices without changing anything about the socio-technical system around them. I described this in 2018 as the red flag law anti-pattern. In some jurisdictions, a pedestrian waving a red flag and blowing a whistle was required to walk in front of an automobile. You have a car that could go faster than a person, but rules and regulations slow it down. That is what exists in a lot of organizations: a go-faster architecture that the organization’s policies, procedures, and structure slow down to pedestrian pace.

The fundamentals matter even more with GenAI

The practices that make software easy to change matter more now than they did before GenAI. A coding agent depends on fast feedback and on guardrails. Both of those come from your architecture and your tests.

Coding agents need fast feedback

A coding agent essentially has to use test-driven development. It needs to be able to quickly run the tests and get feedback, so the inner developer loop has to run exceedingly quickly. The outer loop has to be fast too: a push triggers the deployment pipeline, and that needs to run quickly and get the change into production, where it gets more feedback.

The whole motivation for a fast-flow architecture is rapid feedback, so you want to structure your code, your tests, and your deployment pipelines so that your coding agents get prompt feedback. In that sense, microservices are even more likely to be used today than they were before GenAI.

Fast feedback is a property of the architecture

As I wrote in GenAI-based software delivery needs a fast flow architecture, fast feedback loops are a property of the architecture. Loose design-time coupling keeps the scope of a change as narrow as possible. Loose build-time coupling minimizes how much unchanged code has to be rebuilt and retested as a consequence of a change. And the decision to divide an application into services, each with its own deployment pipeline, is an architectural decision too. So architecture is absolutely essential if you are going to benefit from coding agents.

Agents need guardrails

I have had success with GenAI when there are strong guardrails in place:

Automated tests - one of the first guardrails.
The more boring, deterministic guardrails - pre-commit hooks that run code-quality checks (I use CodeScene) and vulnerability checks, to keep the AI from doing something stupid.

Otherwise it writes code that it never uses. On one project, 30% of the generated code was dead code - which admittedly happens with humans too, but this is meant to be world-changing technology.

Michael Feathers defined legacy code as code without tests. A tremendous amount of the code being written today does not have proper automated tests, so you let agents loose on it and who knows what bugs they are going to introduce.

Software development is still a human activity

There is also a troubling trend: the idea that AI generates code so fast that we can’t review it, so we must find other ways to validate it. Call me old-fashioned, but software development is still a human activity. It is your code, with your name on it, that you are committing and are accountable for. And I do not think coding agents are capable of the good design that human beings are.

Brownfield is not greenfield

Luca asked whether AI can really take a 20-year-old ERP and quickly move it to whatever architecture you want.

GenAI helps you understand legacy code

GenAI is a useful tool for understanding a legacy codebase, though you have to take steps to ensure that it does not hallucinate. I have had it produce architecture documentation that was quite imaginary: half of it was real, and half was how it thought the system should work, even though it does not.

Modernization is more than generating code

GenAI can also generate code, but it needs the guardrails, the tests, and the specification in order to generate the right code. And modernizing an application involves a lot more than generating code. Quite often you are having to reverse-engineer the requirements. You are also having to design how the new application should work, applying modern techniques such as domain-driven design, good boundaries, and modularity. And as you build it, you are having to learn the new technology you are building it with. There is all this learning going on, and you do it incrementally. That is why I much prefer the Strangler Fig pattern, where you migrate iteratively, over a big bang approach where you get no validation until you are done.

Next.js is not your legacy system

There are some well-known examples of rewrites. Someone rewrote Next.js in a week. But Next.js had a very exhaustive test suite, was well documented, and the web is full of content about it. That is not your legacy system.

I see the same thing on one of my side projects, a language interpreter. I can tell Claude Code “implement language feature X” and it goes off and implements it perfectly, because that language feature is incredibly well documented. Whereas when I am doing normal enterprise-style development, it muddles around and it is a whole lot harder to figure out what needs to be done. A legacy application is a million times worse: a 30-year-old application that might even predate the web.

Convince me

I am willing to be convinced that there is finally some magic technology where you press a button and, hey presto, you get a modern, well-architected application within a week. But people have to demonstrate that it works and that it is not just a big-tech smoke-and-mirrors demo, like a lot of stuff is. Convince me.

Get the fundamentals right

Luca asked me what advice I would give architects going into 2026. My answer has not changed much, because the problems have not changed much either.

A 50-year-old idea that still has not caught on

Most enterprises are struggling with the fundamentals. Modularity and loose coupling date back to Parnas in 1972. That is a 50-year-old concept, and it still has not caught on. A change should ideally happen within one team, but a lot of enterprises have built software that requires a massive number of meetings to coordinate and align in order to roll out a really simple change. There is all this new shiny stuff, but software development would be way better if people got the hang of the fundamentals. And maybe automated testing will catch on too.

The job of an architect is to solve problems

My advice is the same as always. The job of an architect is to solve problems, and you need a disciplined way of doing that:

define the problem,
come up with some success criteria,
brainstorm candidate solutions,
evaluate them against the criteria, and
pick the best approach.

Above all, actually think things through.

Demand proof

Have a healthy skepticism. You do not want to be super negative, but you should demand proof: extraordinary claims require extraordinary evidence. Just because some random person on social media says “this worked for me” does not mean it is going to work for you. My mindset is heavily patterns oriented: proven solutions to problems in a given context. Your context is not the same as the random person on X’s context, and you have to carefully think it through.

What else we got into

On the episode, Luca and I also discuss:

why code is not free - human attention is still a limited resource - and why “three commits before breakfast” leads to burnout: a relentless pair programming partner who won’t stop, and you can’t stop either
agents for everything: the same overuse we saw with microservices, and how to decide which sub-problems actually need a non-deterministic agent
“SaaS is dead”, and why big tech is so desperate to make GenAI look revolutionary
why Luca thinks we are all juniors again, and what it feels like to use GenAI without a conceptual framework
whether GenAI is undermining the business model of open source, and whether it is hard to justify hiring junior developers
whether this can even be a profitable industry once you get past Nvidia, the cloud providers, and the consulting companies
whether spec-driven development is a return to waterfall

You can watch the whole conversation on the Dear Architects podcast.

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →

Architecting for uncertainty - part 1: thriving in a VUCA world requires frequent deployment

Tue, 25 Aug 2026 08:03:00 +0000

This is the first in a series of articles based on my talk Architecting for Fast Flow: Thriving Amid Uncertainty. In this series, I define the term fast flow and explain why it has become an essential software delivery capability. I discuss how fast flow emerges from a socio-technical architecture, which consists of a software architecture and its development organization. Finally, I describe how to design and evolve a socio-technical architecture in order to sustain fast flow.

The articles in the series are:

This first article is the motivation for the rest of the series: today’s business environment is turbulent, and businesses must rapidly adapt their products and services. And since modern businesses are powered by software, their ability to adapt depends on how quickly they can change their software.

Earth image: NASA / Artemis II

Earth is quite beautiful. But for its inhabitants it’s increasingly turbulent. Or, to use the language of management theory, it’s VUCA: volatile, uncertain, complex and ambiguous. The result is a world that is harder to predict, understand and navigate. In this article, I describe the impact of VUCA on businesses and explain why thriving in such a world requires frequent experimentation - and hence frequent deployment.

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →

Speed limits, GenAI coding agents and Autobahns - part 2: raising the safe speed

Sat, 22 Aug 2026 08:03:00 +0000

Part 1 developed a driving metaphor for GenAI-based development: you are the driver, the coding agent and its harness are the car, and your code and delivery system are the road. And in the same way that the road determines how fast you can drive, the characteristics of the code and the socio-technical architecture around it determine how fast you should let the agent generate code, and how much cognitive debt, if any, is acceptable.

Part 1 also covered the first of the three ways of working with a coding agent: understand the road - what condition the code and the socio-technical architecture around it are in - which tells you the safe speed. This article describes two ways to raise the speed limit. First, you can improve the road: the architecture, the tests, the guardrails and the deployment pipeline. Second, you can improve the car: the model, the prompts, the tools and the workflow that make up the harness.

Both of these improvements work the same way: they improve the conditions, increasing confidence that the agent’s output will be correct, and confidence is what sets the safe speed. Let’s look at each improvement in turn, starting with the road.

Improve the road: make a stretch safe at higher speed

Unlike a driver, you aren’t stuck with the road as it is. You can improve it: you can decouple the code, tighten the guardrails, limit the damage when something does get through, and speed up the deployment pipeline. What’s more, road improvements are persistent: decouple a module once, and every future change to it is easier.

Let’s look at each way to improve the road in turn.

The road surface: loose design-time coupling

Loose design-time coupling is the road surface. It localizes changes so that both people and agents can work faster, and the agent uses fewer tokens and less context. A tightly coupled - 25 mph - section becomes a faster stretch of road once it’s decoupled.

Guardrails: automated tests and checks

Some improvements catch mistakes and prevent a bad change before it leaves the road: fast automated local tests, type checking, linters, CI, and human review. The more reliably a mistake is caught early, the faster you can safely generate code.

A check that depends on somebody remembering to run it isn’t really a guardrail, so automate it. Pre-commit checks, Git hooks, CI jobs and automated pull request review can gate the change at different points, providing defense in depth.

Run-off areas: canary deployments and fast rollback

A run-off area limits the damage when a mistake does get through. The software equivalents are canary deployments, fast automated rollback and smaller batches. The cheaper a mistake is to recover from, the less it costs to be wrong, and the faster you can safely go.

Extra lanes: microservices with their own deployment pipelines

Just as a road becomes congested when traffic exceeds its capacity, a deployment pipeline becomes a bottleneck when changes arrive faster than it can deploy them. A handful of developers running agents can generate more commits in a day than a much larger team once did. Joe Magerramov’s “The Valley of Calm” describes how at around a hundred commits a day, a pipeline “stops behaving like a pipeline and starts behaving like a traffic jam.” Because batches are cumulative, a defect rejects the whole batch, and the commits that arrive while you fix it make the next batch bigger and more likely to fail again. His simulation shows that beyond a certain point teams end up in a plateau of misery where almost nothing ships.

The remedy for road congestion is more lanes. Similarly, as I describe in GenAI-based software delivery needs a fast flow architecture, the solution for an overloaded deployment pipeline is a microservice architecture. Each service has its own deployment pipeline, which handles fewer commits from a single team and verifies a smaller code base. Collectively, these deployment pipelines can process many more commits than a single shared pipeline.

Improve the car: strengthen the agent and its harness

Now let’s consider the car. Besides its speed, a real car has safety features: high-performance brakes, collision avoidance, seatbelts, and airbags. A coding agent is similar. The raw model sets the top speed, and the harness is what makes it safe to use.

Structurally, a harness is the coding agent sandwich: plain old code orchestrating on top, the model in the middle, and deterministic tools underneath. My own implement agent orchestrator is built that way.

The harness both prevents mistakes and catches them. As Birgitta Böckeler writes in Harness Engineering, a harness consists of guides that shape what the agent does before it acts, and sensors that provide feedback so it can correct course afterward.

Guides: shaping what the agent does before it acts

Guides tell the agent how to work: coding conventions, architecture documents, skills and how-to guides, and the deterministic tooling underneath - e.g. language servers, CLIs, automated refactoring tools - that makes the right thing the easy thing. Keep them in good order so that the agent has what it needs. The better the guides, the more often the agent gets it right the first time, so the less you have to slow down to check its work.

Sensors: providing feedback to the agent

Sensors provide feedback enabling the agent to course-correct: linting, static type checking, dead code detection, test coverage verification, and code health analysis such as CodeScene. I described some of these in a GenAI-based development platform. Those all run against the code, but a sensor can also report from the running system: observability data tells the agent what a change actually did, not just whether the code looks correct.

A check that is a sensor can also be a guardrail, and vice versa. When a check runs in the pipeline, it’s a guardrail that gates the change. And when it runs alongside the agent, it is a sensor that provides feedback so that the agent fixes the problem itself. The better the sensors, the more mistakes the agent catches and fixes before you ever see them, so the more confident you can be in its output, and the faster you can let it go.

In the same way that a better car lets you go faster on most roads without removing the road’s limits, a better harness lets you go faster with a coding agent without removing the limits imposed by the codebase and the team. That’s why it’s important to improve both the road and the car.

Summary

The safe speed isn’t fixed. Improving the road - the architecture, the tests, the guardrails and the deployment pipeline - and improving the car - the harness around the model - both raise it. They increase confidence that the agent’s output will be correct, and confidence is what sets the safe speed.

That’s what fast flow means in practice: moving fast without breaking things. Set the speed limit deliberately, then do the work to raise it.

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →

Microservices Platforms: When Team Topologies Meets Microservices Patterns - QCONSF 2025 video

Wed, 05 Aug 2026 08:03:00 +0000

The video and transcript of my QCON San Francisco 2025 talk on microservices platforms - the basis for the article series - is now available.

Abstract

When many teams work on a large, complex application, the microservice architecture potentially enables them to work independently and deliver a continuous stream of changes. Yet, there’s a risk that those teams become burdened by needing to apply numerous patterns from the microservice architecture pattern language. Instead of writing domain logic, they must spend their time writing plumbing code and dealing with infrastructure. This overhead ultimately slows feature delivery, undermining the primary goals of adopting a microservice architecture.

In this talk, I explain how to solve this problem by applying the Team Topologies’ concept of a platform, which reduces the workload for teams and enables them to focus on domain logic. You will learn how this platform approach directly tackles the most common challenges of implementing the microservice pattern language. I will then describe several kinds of platforms and the key capabilities they provide to free development teams from this implementation burden.

Articles

Here’s the series of articles that I wrote based on this talk:

Slides

See the slides.

Video and transcript

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →

Speed limits, GenAI coding agents and Autobahns - part 1: how fast should you go?

Tue, 04 Aug 2026 08:03:00 +0000

GenAI coding agents enable a team to generate code rapidly. However, each team member is powered by a 20 W organic processing substrate, and there’s a limit to the rate at which they can read and understand code. As a result, the team cannot keep up, creating a serious problem: cognitive debt. Code gets committed and deployed that nobody understands. The gap between what the system does and what the team knows it does keeps widening.

But it does not have to be this way. Just because an agent can generate code faster than a team can understand it doesn’t mean a team should let it. The question every team should be asking is:

How fast should we let the agent go on our code?

My sense, however, is that not enough teams are asking this question.

This article is the first of two about how a team can answer this question. In this part, I’ll explain why you can slow down and avoid creating cognitive debt, and then introduce the metaphor I use in both articles: a fast car. I give the three reasons I don’t drive my car at its top speed, and what each of them corresponds to in software. I then deal with the two obvious counter-examples: the Autobahn, which has stretches with no posted speed limit, and Formula 1, where maximum speed is the entire point. Finally, I describe how to work out the speed that’s safe for your code. Part 2 covers how to increase the speed limit. Let’s start by looking at why we have a choice about how fast to let a coding agent go.

You choose how fast the agent goes

Rapid code generation that exceeds human understanding is usually presented as inevitable. And dealing with it is an active research topic, with AI-assisted code review and better tooling as examples. But usually the generation rate is taken as given and work focuses on helping teams keep up.

In fact, the rate at which an agent can generate code isn’t the rate a team has to accept. Teams have control over the rate of code generation and the amount of cognitive debt they create. To understand why, let’s look at the metaphor of a fast car.

Why I don’t drive at my car’s top speed

This metaphor consists of the following elements:

You are the driver.
The car is the coding agent, together with the harness around it.
The road is the code, and the socio-technical architecture.
Speed is the rate at which code is generated and deployed.

Like a coding agent, my car is fast. According to the specs, it has a top speed of around 150 mph. But I don’t drive that fast. Let’s look at the three reasons why.

On many roads it isn’t safe

First, it’s often unsafe. I could injure or kill myself, or someone else. Rapidly generating code when understanding is required - core logic, novel design, etc. - means deploying code the team can’t reason about. That’s cognitive debt.

Speed limits

Second, traffic laws require me to observe the speed limit, typically 25 to 65 mph. I could ignore them, but I’d risk a ticket. With code there’s no highway patrol, but a team can choose its own limits: for example, that a team member must understand a change before it merges.

But the road ahead is jammed

Third, there’s congestion. The car’s top speed is irrelevant when the road is full. For instance, the average traffic speed in many major cities is under 15 mph!

As I describe in part 2, without the right architecture, a coding agent can generate more changes than the deployment pipeline can process. If the deployment pipeline is overwhelmed, faster code generation produces fewer successful deployments, not more.

But what about the Autobahn?

There are, of course, places where you can drive much faster. The Autobahn has stretches with no posted speed limit. But drivers are legally required to adjust their speed to the traffic, weather, visibility and road conditions. And there’s an advisory 130 km/h (81 mph) that can count against them in a liability claim if they crash above it.

The Autobahn is a safe road system for the following reasons:

The permitted speed is matched to the conditions. Permanent limits where the road demands them, variable limits where conditions change, and unrestricted stretches only where the design allows.
The road is engineered to support higher speeds safely. The surface, the shoulders, the banking, and the guardrails.
Vehicles and drivers are held to a high standard. Demanding licensing, and mandatory periodic inspection of every car.

The unrestricted stretches are unrestricted because the conditions there allow it, and a limit appears on the same road on the same day when the conditions change. It’s a great example of matching the speed limit to the conditions.

And then there’s F1

Formula 1 might appear to be another example of how to go fast. The custom-designed cars are far faster and are driven at their limit. But the drivers are professionals with teams monitoring telemetry. Also, there are only two dozen circuits. The drivers know each one intimately and rehearse for hours on simulators before the race. Part of what makes the speed safe is that nothing about the track is a surprise.

But software development involves novelty. There’s no equivalent of simulator time or practice sessions before the race. In fact, if an application were sufficiently standardized, you’d be better off buying it than building it. So F1 isn’t a suitable metaphor for software development.

Understand the road, improve the road, improve the car

Now that we’ve explored the metaphor of a fast car, let’s return to the question of how fast to let a coding agent go. Three aspects of the Autobahn’s safety map onto three ways to work with a coding agent:

Understand the road - determine the coding agent’s safe speed for each module of your application
Improve the road - build the infrastructure that enables the coding agent to work safely at higher speed
Improve the car - improve the agent and its harness.

Part 2 describes how to improve the road and the car. The rest of this article looks at how to understand the road.

Understand the road: how fast can you drive it?

Understanding the road tells you what speed is safe. It’s how you recognize a school zone or a winding mountain road and slow down before you get there. Similarly, the characteristics of the software determine the coding agent’s safe speed.

Here are some per-module characteristics that indicate the safe speed for a coding agent:

DDD, core versus everything else. The core domain is a 25 mph zone. Supporting and generic subdomains are usually open road, and defining those subdomains is what tells you which is which.
Hexagonal, domain versus adapters. Adapters and glue code are usually open road. The domain model rarely is.
Coupling and testability. Loosely coupled, locally testable code is open road, because changes stay contained and fast tests catch mistakes. Tightly coupled, hard-to-test code is a 25 mph zone regardless of anything else.
Cynefin: clear, complicated or complex. Clear and complicated work - well-established patterns the model has seen many times - allows speed, because a known good practice applies. Complex work, where novel architecture has no precedent, does not: you have to probe and learn, so slow down.

Two more characteristics apply to the application as a whole rather than to individual modules:

Application scale and purpose. A throwaway prototype differs from a long-lived production system, and low blast radius differs from high.
GenAI capability. How well represented is this work in the model’s training? A common stack allows speed. An obscure or bespoke one does not.

Together, those characteristics determine how much confidence you can have that the agent’s output will be correct. It is this confidence that sets the safe speed, as the diagram above illustrates. Where confidence is high, let the agent go faster. Where it’s low, slow down and review.

And a road trip is rarely all one kind of road. It might start on suburban streets, join the freeway for two hundred miles, climb a winding mountain pass, and finish in a school zone. The safe speed changes several times along the way, and a good driver adjusts each time.

An application is the same. The per-module characteristics vary from one module to the next, and so does the confidence they give you. The safe speed is therefore mostly a property of the module you’re changing, rather than a single number for the whole application. Implementing a single feature can involve modules with very different safe speeds, so expect to slow down and speed up as you go.

Summary

The road you’re on determines how fast you can drive: an Autobahn, a winding mountain road, a school zone, etc. Code is the same. The characteristics of the codebase and the team determine how fast you should let the agent generate code, and how much cognitive debt, if any, is acceptable. And just as a road trip covers several kinds of road, an application contains several kinds of code, each with its own safe speed.

But unlike a driver, who takes the road and the car as they are, you shape both. Part 2 covers how to raise the safe speed: improve the road, and improve the car.

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →

Q&A about my Explore DDD designing microservices workshop

Mon, 20 Jul 2026 08:03:00 +0000

I’m teaching an updated version of my workshop - Designing microservices: responsibilities, APIs and collaborations - at the excellent Explore DDD conference that’s happening this September in Denver.

Here’s a Q&A that explores why this workshop is especially relevant in the age of GenAI, the common mistakes teams make when designing and developing microservices, and how the workshop equips attendees with a practical, repeatable approach to building microservice architectures that enable fast, sustainable software delivery.

What’s changed most about microservices design in recent years?

The recent surge in adoption of GenAI coding agents has strengthened the case for the microservice architecture. That’s because coding agents accelerate the flow of changes, which in turn demands an architecture that can support that increased flow rate. As a result, organizations will outgrow the monolithic architecture much sooner. What’s more, a service’s smaller codebase improves the effectiveness of coding agents.

And what’s stayed stubbornly the same?

Designing a microservice architecture is still high stakes and challenging. A bad architecture will slow down delivery and threaten the success of the business. Yet at the same time, defining an architecture requires deep thinking to navigate numerous tradeoffs.

Where does AI genuinely help in designing services, and where does it often confidently steer someone wrong?

GenAI is a helpful design assistant. It can extract domain models and APIs from requirements, suggest service decompositions, identify responsibilities, generate APIs, and rapidly explore alternatives. However, despite appearances to the contrary, GenAI is a next-token predictor that understands nothing and cannot reason. It can define a superficially plausible yet deeply flawed architecture - for example, one that looks reasonable at a glance but bakes in tight coupling or the wrong service boundaries. Human architects are therefore still essential: they define the architectural guardrails that guide GenAI, and they exercise the judgment needed to make the architectural decisions.

What mistake(s) do teams most often make when they carve a system into microservices?

I regularly encounter organizations that have built an excessively fine-grained service architecture, where each small team owns many services - e.g., one service per team member, or worse. Such architectures are difficult to understand and maintain, and there’s often tight design-time and runtime coupling. What’s more, instead of solving business problems, teams are forced to deal with distributed systems issues such as complex coordination logic.

What does a day in this workshop actually look like, hands-on?

While there are lectures, the core of the workshop - and the most fun part - is the design exercises and the discussions. The day alternates between short lectures that introduce concepts and exercises that immediately put them into practice, with attendees working together in small groups to design a microservice architecture. After each design exercise, there are great discussions comparing the different groups’ designs, debating the tradeoffs, and discovering that there’s rarely one right answer.

What will attendees be able to bring back to their teams to help them succeed with microservices?

Attendees leave with several concrete takeaways:

A deep understanding of key microservice architecture concepts, including when - and when not - to use microservices.
A practical, repeatable approach for defining a microservice architecture with the appropriate level of granularity.
Techniques for designing services that are loosely coupled at both design-time and runtime.
How to apply the service collaboration patterns.

To learn more

To learn more about the workshop, and to register, visit the Explore DDD conference website. And, if you would like me to teach this workshop at your organization, please contact me.

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →

Migrating a Claude Code skill from WebFetch to the new CircleCI CLI: lessons learned about tool and API design

Fri, 17 Jul 2026 08:03:00 +0000

One of the Claude Code skills in my idea-to-code plugin is debugging-ci-failures. It tells Claude Code how to watch and diagnose a failing CI build. Specifically, it instructs Claude Code to do the following five steps:

Watch the build to completion
Identify the failing job
Fetch the failing step’s output
Download the test artifacts
Read the JUnit TEST-*.xml files to find the root cause

The skill supports two CI systems: GitHub Actions (via the gh CLI) and CircleCI. Until today, the CircleCI support used WebFetch — Claude Code’s built-in HTTP fetch tool — to access CircleCI’s REST API directly. That worked, but it was clumsy.

This post is about how I used Claude Code to migrate the skill from WebFetch to the new CircleCI CLI. The migration itself took a single one-line prompt — but that’s the least interesting part. Along the way, Claude Code discovered CLI features I didn’t know existed and found a shortcut I hadn’t thought to ask for. And by the end, a small skill update had turned into something more general: a lesson about how we should be designing tools for AI agents. I’ll come back to that. First, the hack the new CLI let me delete.

The WebFetch approach and its ugly workaround

Compared to the GitHub gh CLI, the existing CircleCI CLI was rather limited. As a result, the skill had to use WebFetch to access CircleCI’s REST API directly, which was rather inconvenient. Not only did Claude Code need to know the details of the REST API, but analyzing builds for private repositories required an API token, which was not something I wanted to expose to Claude Code.

Also, the Claude Code WebFetch tool caches responses for about 15 minutes, which is a problem when you are invoking an API. As a workaround, the skill had to append a cache-busting query parameter to every request:

WebFetch: https://circleci.com/api/v1.1/project/github/<org>/<repo>?limit=1&branch=<branch>&_ts=1

Note the &_ts=1 at the end. That’s not a CircleCI parameter — it’s a hack that forces WebFetch to treat each request as unique.

The new CircleCI CLI

I was excited to read Rob Zuber’s LinkedIn post announcing the new CircleCI CLI. A quick glance showed that it was roughly comparable to the gh CLI for GitHub Actions — it handled authentication, and hid the details of the REST API. What’s more, I discovered that changing the skill to use it was remarkably straightforward.

The prompt that started the migration

After I reorganized the skill so that GitHub Actions and CircleCI each had their own section, my next prompt was a single line:

For circleci some or all of the webfetch can be replaced by the new circleci CLI. For example, circleci run list — you can try on eventuate-foundation/eventuate-common.

Claude Code walked the CLI’s help tree (circleci --help, circleci run --help, circleci job --help, etc.), then ran commands against eventuate-foundation/eventuate-common to see the actual JSON shapes and behaviors before rewriting the skill.

Take a look at the Claude Code session and the resulting commit.

What the CLI replaces

The mapping turned out to be direct — the CLI has a purpose-built command for every step of the workflow:

Skill step	Old (WebFetch + `_ts` hack)	New (`circleci` CLI)
Watch a build	Poll `/build/<n>?_ts=N` in a loop	`circleci run watch` — blocks, exit code = outcome
Find the latest run	WebFetch project endpoint	`circleci run list --branch <branch>`
Get run + workflow + job status	WebFetch, parse JSON	`circleci run get <run-id> --json`
Find the failed step	WebFetch `?include=steps`, scan	`circleci job output list <job-id> --json --jq '.steps[] \\| select(.exit_code != 0)'`
Fetch the failed step’s output	WebFetch `/output/<idx>/0`	`circleci job output get <job-id> --step-num N`
List artifacts	WebFetch `/artifacts`	`circleci artifact <job-id> --json`
Download artifacts	WebFetch each URL individually	`circleci artifact <job-id> --output test-reports`

Two of these are worth highlighting.

`circleci run watch`

circleci run watch replaces an entire polling loop with a blocking call whose exit code encodes the outcome:

0 — all workflows succeeded
1 — one or more workflows failed
6 — cancelled
8 — timed out

That’s a much better contract for an agent than “keep polling until status is no longer running, then check outcome.”

`circleci job output get --condensed`

Claude discovered that the circleci job output get command has a rather useful flag: --condensed:

Use –condensed to fetch a filtered version of stdout only, with noisy or repetitive lines removed server-side. This produces a much smaller payload suitable for feeding to an AI tool. When –condensed is set, output is always rendered through the plain-text renderer regardless of the terminal or –strip-ansi.

CircleCI has apparently added a first-class “give me the interesting lines, meant to be pasted into an LLM” mode to their CLI.

An unexpected shortcut

While Claude Code studied the CLI’s commands, it found another one that isn’t strictly a WebFetch replacement — it’s an entirely new step Claude can try before downloading anything:

circleci testresult list <job-id>

By default, this command shows a job’s failed tests. It enables Claude Code to quickly identify the failing test without downloading any artifacts. The skill now tries testresult list first and falls back to downloading TEST-*.xml files only if that isn’t enough.

Why this matters beyond one skill

The bigger point isn’t about CircleCI or Claude Code specifically. There are a few broad lessons here:

Prefer tools to raw APIs
Design CLI tools to be discoverable
Design CLI tools for token-efficient output

Let’s look at each one.

Prefer tools to raw APIs

If your agent has to reach an external system through a general-purpose HTTP tool, it’s writing REST calls by hand, dealing with caching, composing URLs, and parsing JSON shapes. Every one of those steps is a place the agent can go wrong.

If instead the agent can reach the same system through a well-designed CLI, the CLI hides all of that. circleci run watch doesn’t need cache-busting. circleci artifact --output doesn’t need URL composition. circleci job output get --condensed doesn’t need the agent to figure out how to trim noise for an LLM — the vendor already did.

A CLI aimed at agents is a shorter path than the raw API, and shorter paths are more reliable. When you’re writing skills, prefer the CLI when one exists — and let the absence of one push you to open an issue with the vendor rather than to hand-craft another URL.

Design CLI tools to be discoverable in the era of shallow understanding

It’s also worth pointing out that Claude Code figured out how to use the CircleCI CLI without me telling it anything about the commands. It simply ran circleci --help, then circleci run --help, then circleci job --help, and so on, until it had a mental model of the CLI’s command tree. That was enough for it to determine the right commands and update the skill. If you are developing CLI tools, make sure they are discoverable in this way — the --help output is a first-class part of the interface, not just a convenience for humans.

Design CLI tools for token-efficient output

Finally, the --condensed flag is a great example of a CLI designed for an agent’s needs. Filtering noise server-side lowers token cost and keeps the agent’s context window focused on what matters — and the vendor knows which lines are noise better than any skill author does. Worth considering whenever you design an API or CLI tool.

About this article

Claude Code wrote the first draft of this post during the same session that made the change. The skill migration and the article came out of one conversation. First, I asked Claude to migrate the skill to the CircleCI CLI tool. After that, I told it to draft this post.

Need help with modernizing your architecture?

I help organizations modernize their architecture to enable fast flow and GenAI-powered software delivery. If you’re planning or struggling with a modernization effort, I can help.

Learn more about my modernization and architecture advisory work →