multi-architecture docker images   docker  

Building multi-architecture Docker images for Intel and ARM

This is the second article about my adventures trying to use my Apple M1 MacBook for development. The first article described how I need to change the Eventuate platform and the example application to build and/or use multi-architecture Docker images that support both Intel and ARM. Since that involves changing numerous projects that have non-trivial builds, I decided to start with a project that builds a container image for much simpler Java application. This article describes what I learned building a multi-architecture Docker image that runs PlantUML, which is an incredibly useful UML diagramming tool.

The other articles in this series are:

About the PlantUML project

I use PlantUML together with AsciiDoc to create all kinds of documents. PlantUML is written in Java but relies on GraphViz, which is C-based graph visualization package. To run PlantUML locally without having to worry about installation issues, I created the microservice-canvas/plantuml project. It builds a Docker image that packages PlantUML along with its GraphViz dependency.

The Docker image reads a text-based UML diagram from stdin and writes a png to stdout. For example, to generate a PDF from an AsciiDoc file containing PlantUML-generated images I typically use a Makefile that contains rules like this:

%.pdf : %.adoc
  asciidoctor -r asciidoctor-pdf -b pdf -o $@ $<

%.png : %.txt
   docker run -i --rm --net=none microservicesio/plantuml:0.2.0.RELEASE < $< > $@

Building the PlantUML image

The Dockerfile that builds the image is quite simple:

FROM amazoncorretto:8u312-al2
WORKDIR /plantuml
RUN yum -y install graphviz-2.30.1 imagemagick wget && \
    yum clean all && \
    rm -rf /var/cache/yum
RUN wget https://github.com/plantuml/plantuml/releases/download/v1.2021.16/plantuml-1.2021.16.jar
CMD ./run-plantuml.sh
...

The Dockerfile installs some dependencies, and downloads the PlantUML JAR file.

When I run this Intel-only image on my M1 MacBook, I get a WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested. The image runs very slowly and sometimes hangs or crashes. Clearly, QEMU-based emulation does not work well.

Let’s now look at how to a build multi-architecture image

Building a multi-architecture image using docker buildx build

On the surface, building a multi-architecture image is easy. You simply replace the docker build with the docker buildx build command:

docker buildx build --platform linux/amd64,linux/arm64 -t microservicesio/plantuml:test-local ...

The docker buildx build builds Docker images using BuildKit, which is a toolkit for building and packaging software. The --platform option specifies the target architectures.

Note: before running docker buildx build for the first time, you have to create a builder. For example, the docker buildx create --use command creates a randomly named builder with the default configuration settings. The --use option makes it the current builder.

Seems simple, right? Not quite.

The base image must be multi-architecture

The first issue is that in order for docker buildx build to create a multi-architecture image, the Dockerfile must specify a multi-architecture base image. Fortunately, the amazoncorretto:8u312-al2 used by this project supports both Intel and ARM so I didn’t need to search for a different image. But as I will describe in a later article, it’s not always straightforward to find a suitable base image.

Where does docker buildx build put the image?

Another issue with using docker buildx build is that, unlike docker build, it doesn’t create a local container image within the Docker daemon. That’s because the Docker daemon can only contain single architecture images for the platform that it’s running on. Instead, docker buildx build must either push the image to a registry or write it to the filesystem.

Pushing the image to a registry seems like a much better choice. But which registry? For now, a locally running Docker registry container seems like a good choice so I created a simple docker-compose.yml file:

#! /bin/bash -e
version: '3'
services:
  registry:
    image: registry:2
    ports:
      - "5002:5000"

Note: I used port 5002 because 5000 conflicted with something already running on my MacBook.

Using docker buildx build with a local container registry

Once I started the registry container, I thought I simply run docker buildx build with the --push option:

docker buildx build \
   --platform linux/amd64,linux/arm64 \
   -t localhost:5002/plantuml:test-build \
   --push ...

Sadly, it’s not that simple for two reasons.

First, the BuildKit builder is a container so localhost doesn’t resolve to the host machine. The solution is to use host.docker.internal as the host name.

docker buildx build --platform linux/amd64,linux/arm64 -t host.docker.internal:5002/plantuml:test-build --push ...

host.docker.internal is a special DNS name that resolves to a host IP address.

The second problem is that docker buildx build expects to push to a secure, HTTPS-based registry. The solution is to replace the shorthand --push with the longer --output=type=image,push=true,registry.insecure=true:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t host.docker.internal:5002/plantuml:test-build \
  --output=type=image,push=true,registry.insecure=true \
  ...

Once I made this change, the docker buildx build successfully built the image and pushed it to the registry. I can then run the image using docker run -i --rm localhost:5002/plantuml:test-build.

Inspecting the image’s manifest

So far docker buildx build has created what is supposedly a multi-architecture image. However, I haven’t yet ran the same image on both Intel and ARM machines. To do that I need to push the image to a remote registry that’s accessible by multiple machines. I describe a deployment pipeline that does this in the next article.

In the meantime, we can use the docker manifest inspect to list an image’s supported architectures. Let’s first inspect the original microservicesio/plantuml:0.2.0.RELEASE image.

$ docker manifest inspect --verbose microservicesio/plantuml:0.2.0.RELEASE
{
	"Ref": "docker.io/microservicesio/plantuml:0.2.0.RELEASE",
	"Descriptor": {
		"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
		"digest": "sha256:...",
		"size": 3253,
		"platform": {
			"architecture": "amd64",
			"os": "linux"
		}
	},
	"SchemaV2Manifest": {
		"schemaVersion": 2,
		"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
		"config": {
			"mediaType": "application/vnd.docker.container.image.v1+json",
			"size": 6296,
			"digest": "sha256:.."
		},
		"layers": [ ... ]
	}
}

It returns a JSON object describing the image architecture - amd64 - and its layers.

Next, let’s inspect the localhost:5002/plantuml:test-build image. The output is quite different. Instead of a JSON object, the docker manifest inspect command outputs a JSON array with one element for each architecture.

$ docker manifest inspect --verbose --insecure localhost:5002/plantuml:test-build
[
	{
		"Ref": "localhost:5002/plantuml:test-build@sha256:...",
		"Descriptor": {
			"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
			"digest": "sha256:...",
			"size": 1994,
			"platform": {
				"architecture": "amd64",
				"os": "linux"
			}
		},
		"SchemaV2Manifest": {
      ...
			},
			"layers": [ ... ]
		}
	},
	{
		"Ref": "localhost:5002/plantuml:test-build@sha256:...",
		"Descriptor": {
			"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
			"digest": "sha256:...",
			"size": 1994,
			"platform": {
				"architecture": "arm64",
				"os": "linux"
			}
		},
		"SchemaV2Manifest": {
      ...
			"layers": [ ... ]
		}
	}
]

Note: use --insecure to access an HTTP-based registry

The manifest says that this image supports both Intel/AMD and ARM. This looks promising!

Viewing the changes

To see the changes I made to the project, take a look at this Github commit.


multi-architecture docker images   docker  


Copyright © 2023 Chris Richardson • All rights reserved • Supported by Kong.

About Microservices.io

Microservices.io is brought to you by Chris Richardson. Experienced software architect, author of POJOs in Action, the creator of the original CloudFoundry.com, and the author of Microservices patterns.

Chris helps clients around the world adopt the microservice architecture through consulting engagements, and training workshops.

Learn how to create a service template and microservice chassis

Take a look at my Manning LiveProject that teaches you how to develop a service template and microservice chassis.

New virtual bootcamp: Distributed data patterns in a microservice architecture

My virtual bootcamp, distributed data patterns in a microservice architecture, is now open for enrollment!

It covers the key distributed data management patterns including Saga, API Composition, and CQRS.

It consists of video lectures, code labs, and a weekly ask-me-anything video conference repeated in multiple timezones.

The regular price is $395/person but use coupon JUNVCEJE to sign up for $195 (valid until February 1st, 2023). There are deeper discounts for buying multiple seats.

Learn more

Signup for the newsletter


LEARN about microservices

Chris offers numerous resources for learning the microservice architecture.

Training classes

Chris teaches comprehensive workshops, training classes and bootcamps for executives, architects and developers to help your organization use microservices effectively.

Avoid the pitfalls of adopting microservices and learn essential topics, such as service decomposition and design and how to refactor a monolith to microservices.

Delivered in-person and remotely.


Get the book: Microservices Patterns

Read Chris Richardson's book:

Example microservices applications

Want to see an example? Check out Chris Richardson's example applications. See code

BUILD microservices

Ready to start using the microservice architecture?

Consulting services

Engage Chris to create a microservices adoption roadmap and help you define your microservice architecture,


The Eventuate platform

Use the Eventuate.io platform to tackle distributed data management challenges in your microservices architecture.

Eventuate is Chris's latest startup. It makes it easy to use the Saga pattern to manage transactions and the CQRS pattern to implement queries.

ASSESS your architecture

Assess your application's microservice architecture and identify what needs to be improved.

Consulting services

Engage Chris to conduct an architectural assessment.



Join the microservices google group

Topics

Note: tagging is work-in-process

anti-patterns   ·  application api   ·  application architecture   ·  architecting   ·  architecture documentation   ·  dark energy and dark matter   ·  deployment   ·  development   ·  devops   ·  docker   ·  implementing commands   ·  implementing queries   ·  inter-service communication   ·  loose coupling   ·  microservice architecture   ·  microservice chassis   ·  microservices adoption   ·  microservicesio updates   ·  multi-architecture docker images   ·  observability   ·  pattern   ·  refactoring to microservices   ·  resilience   ·  sagas   ·  security   ·  service api   ·  service collaboration   ·  service design   ·  service discovery   ·  service granularity   ·  service template   ·  software delivery metrics   ·  success triangle   ·  team topologies   ·  transaction management   ·  transactional messaging

All content


Posts

24 Jul 2017 » Revised data patterns