Building multi-architecture Docker images for Intel and ARM

This is the second article about my adventures trying to use my Apple M1 MacBook for development. The first article described how I need to change the Eventuate platform and the example application to build and/or use multi-architecture Docker images that support both Intel and ARM. Since that involves changing numerous projects that have non-trivial builds, I decided to start with a project that builds a container image for much simpler Java application. This article describes what I learned building a multi-architecture Docker image that runs PlantUML, which is an incredibly useful UML diagramming tool.

The other articles in this series are:

• Part 1 - My Apple M1 MacBook: lots of cores, memory and crashing containers
• Part 3 - Configuring a CircleCI-based pipeline to build multi-architecture Docker images
• Part 4 - Testing an Intel and Arm multi-architecture Docker image on CircleCI
• Part 5 - Configuring CircleCI to publish a multi-architecture Docker image
• Part 6 - Developing the Eventuate Common library on an M1/Arm MacBook
• Part 7 - Configuring CircleCI to build, test and publish multi-architecture images for Eventuate Common
• Part 8 - Building a multi-architecture Docker image for Apache Kafka
• Part 9 - Publishing multi-architecture base images for services
• Part 10 - Publishing a multi-architecture Docker image for the Eventuate CDC service
• Part 11 - The Eventuate Tram Customers and Orders example now runs on Arm/M1 MacBook!!

About the PlantUML project

I use PlantUML together with AsciiDoc to create all kinds of documents. PlantUML is written in Java but relies on GraphViz, which is C-based graph visualization package. To run PlantUML locally without having to worry about installation issues, I created the microservice-canvas/plantuml project. It builds a Docker image that packages PlantUML along with its GraphViz dependency.

The Docker image reads a text-based UML diagram from stdin and writes a png to stdout. For example, to generate a PDF from an AsciiDoc file containing PlantUML-generated images I typically use a Makefile that contains rules like this:

%.pdf : %.adoc
  asciidoctor -r asciidoctor-pdf -b pdf -o $@ $<

%.png : %.txt
   docker run -i --rm --net=none microservicesio/plantuml:0.2.0.RELEASE < $< > $@

Building the PlantUML image

The Dockerfile that builds the image is quite simple:

FROM amazoncorretto:8u312-al2
WORKDIR /plantuml
RUN yum -y install graphviz-2.30.1 imagemagick wget && \
    yum clean all && \
    rm -rf /var/cache/yum
RUN wget https://github.com/plantuml/plantuml/releases/download/v1.2021.16/plantuml-1.2021.16.jar
CMD ./run-plantuml.sh
...

The Dockerfile installs some dependencies, and downloads the PlantUML JAR file.

When I run this Intel-only image on my M1 MacBook, I get a WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested. The image runs very slowly and sometimes hangs or crashes. Clearly, QEMU-based emulation does not work well.

Let’s now look at how to a build multi-architecture image

Building a multi-architecture image using docker buildx build

On the surface, building a multi-architecture image is easy. You simply replace the docker build with the docker buildx build command:

docker buildx build --platform linux/amd64,linux/arm64 -t microservicesio/plantuml:test-local ...

The docker buildx build builds Docker images using BuildKit, which is a toolkit for building and packaging software. The --platform option specifies the target architectures.

Note: before running docker buildx build for the first time, you have to create a builder. For example, the docker buildx create --use command creates a randomly named builder with the default configuration settings. The --use option makes it the current builder.

Seems simple, right? Not quite.

The base image must be multi-architecture

The first issue is that in order for docker buildx build to create a multi-architecture image, the Dockerfile must specify a multi-architecture base image. Fortunately, the amazoncorretto:8u312-al2 used by this project supports both Intel and ARM so I didn’t need to search for a different image. But as I will describe in a later article, it’s not always straightforward to find a suitable base image.

Where does docker buildx build put the image?

Another issue with using docker buildx build is that, unlike docker build, it doesn’t create a local container image within the Docker daemon. That’s because the Docker daemon can only contain single architecture images for the platform that it’s running on. Instead, docker buildx build must either push the image to a registry or write it to the filesystem.

Pushing the image to a registry seems like a much better choice. But which registry? For now, a locally running Docker registry container seems like a good choice so I created a simple docker-compose.yml file:

#! /bin/bash -e
version: '3'
services:
  registry:
    image: registry:2
    ports:
      - "5002:5000"

Note: I used port 5002 because 5000 conflicted with something already running on my MacBook.

Using docker buildx build with a local container registry

Once I started the registry container, I thought I simply run docker buildx build with the --push option:

docker buildx build \
   --platform linux/amd64,linux/arm64 \
   -t localhost:5002/plantuml:test-build \
   --push ...

Sadly, it’s not that simple for two reasons.

First, the BuildKit builder is a container so localhost doesn’t resolve to the host machine. The solution is to use host.docker.internal as the host name.

docker buildx build --platform linux/amd64,linux/arm64 -t host.docker.internal:5002/plantuml:test-build --push ...

host.docker.internal is a special DNS name that resolves to a host IP address.

The second problem is that docker buildx build expects to push to a secure, HTTPS-based registry. The solution is to replace the shorthand --push with the longer --output=type=image,push=true,registry.insecure=true:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t host.docker.internal:5002/plantuml:test-build \
  --output=type=image,push=true,registry.insecure=true \
  ...

Once I made this change, the docker buildx build successfully built the image and pushed it to the registry. I can then run the image using docker run -i --rm localhost:5002/plantuml:test-build.

Inspecting the image’s manifest

So far docker buildx build has created what is supposedly a multi-architecture image. However, I haven’t yet ran the same image on both Intel and ARM machines. To do that I need to push the image to a remote registry that’s accessible by multiple machines. I describe a deployment pipeline that does this in the next article.

In the meantime, we can use the docker manifest inspect to list an image’s supported architectures. Let’s first inspect the original microservicesio/plantuml:0.2.0.RELEASE image.

$ docker manifest inspect --verbose microservicesio/plantuml:0.2.0.RELEASE
{
	"Ref": "docker.io/microservicesio/plantuml:0.2.0.RELEASE",
	"Descriptor": {
		"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
		"digest": "sha256:...",
		"size": 3253,
		"platform": {
			"architecture": "amd64",
			"os": "linux"
		}
	},
	"SchemaV2Manifest": {
		"schemaVersion": 2,
		"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
		"config": {
			"mediaType": "application/vnd.docker.container.image.v1+json",
			"size": 6296,
			"digest": "sha256:.."
		},
		"layers": [ ... ]
	}
}

It returns a JSON object describing the image architecture - amd64 - and its layers.

Next, let’s inspect the localhost:5002/plantuml:test-build image. The output is quite different. Instead of a JSON object, the docker manifest inspect command outputs a JSON array with one element for each architecture.

$ docker manifest inspect --verbose --insecure localhost:5002/plantuml:test-build
[
	{
		"Ref": "localhost:5002/plantuml:test-build@sha256:...",
		"Descriptor": {
			"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
			"digest": "sha256:...",
			"size": 1994,
			"platform": {
				"architecture": "amd64",
				"os": "linux"
			}
		},
		"SchemaV2Manifest": {
      ...
			},
			"layers": [ ... ]
		}
	},
	{
		"Ref": "localhost:5002/plantuml:test-build@sha256:...",
		"Descriptor": {
			"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
			"digest": "sha256:...",
			"size": 1994,
			"platform": {
				"architecture": "arm64",
				"os": "linux"
			}
		},
		"SchemaV2Manifest": {
      ...
			"layers": [ ... ]
		}
	}
]

Note: use --insecure to access an HTTP-based registry

The manifest says that this image supports both Intel/AMD and ARM. This looks promising!

To see the changes I made to the project, take a look at this Github commit.