Configuring CircleCI to publish a multi-architecture Docker image

This is the fifth article about my adventures trying to use my Apple M1 MacBook for development. In the previous article, I described how to configure a CircleCI-based CI/CD pipeline to test a Docker image on both Intel and ARM platforms. Currently, however, the pipeline doesn’t do anything with the image once the Arm-platform tests pass. It leaves the image in Docker Hub with the tag test-build-${CIRCLE_SHA1?}. In this article, I describe how to configure the pipeline to add a ‘proper’ tag (e.g. BUILD-*, x.y.z.RELEASE, etc) to the image without rebuilding it.

The other articles in this series are:

• Part 1 - My Apple M1 MacBook: lots of cores, memory and crashing containers
• Part 2 - Building multi-architecture Docker images for Intel and ARM
• Part 3 - Configuring a CircleCI-based pipeline to build multi-architecture Docker images
• Part 4 - Testing an Intel and Arm multi-architecture Docker image on CircleCI
• Part 6 - Developing the Eventuate Common library on an M1/Arm MacBook
• Part 7 - Configuring CircleCI to build, test and publish multi-architecture images for Eventuate Common
• Part 8 - Building a multi-architecture Docker image for Apache Kafka
• Part 9 - Publishing multi-architecture base images for services
• Part 10 - Publishing a multi-architecture Docker image for the Eventuate CDC service
• Part 11 - The Eventuate Tram Customers and Orders example now runs on Arm/M1 MacBook!!

Defining a deploy job

The pipeline’s build job is still running the original deploy-artifacts.sh script which pushes the single architecture image to Docker Hub before testing the multi-architecture image on Arm. This no longer makes sense and so the first change is to define deploy job, which only runs deploy-artifacts.sh if the test-arm64 job succeeds:

...
deploy:
  docker:
    - image: cimg/base:stable
  working_directory: ~/plantuml
  steps:
    - checkout
    - setup_remote_docker:
        version: 20.10.11
    - run:
        command: ./deploy-artifacts.sh
workflows:
version: 2.1
build-test-and-deploy:
  jobs:
    - build
    - test-arm64:
        requires:
          - build
    - deploy:
        requires:
          - test-arm64

The deploy job executes once the test-arm64 jobs succeeds. Let’s now look at how to publish the multi-architecture with the correct tag.

Publishing the image with the desired tag

The original deploy-artifacts.sh script simply pushed the locally built image to Docker Hub that had a tag corresponding to the Git branch. For example, a master branch build would push a BUILD-${CIRCLE_BUILD_NUM?} tag, a x.y.z.RELEASE branch build would push a x.y.z.RELEASE tag. What’s different about building a multi-architecture image is that it’s already been pushed to Docker Hub with a test-build-${CIRCLE_SHA1?} tag.

I naively thought that I could assign the correct tag by simply executing the following sequence of commands:

1. docker pull
2. docker tag to add a desired tag
3. docker push

These commands successfully publish an appropriately tagged image. However, that image is an Intel-only image! Yet another reminder that a Docker daemon only supports a single architecture. Pulling a multi-architecture image only downloads an image for that architecture.

One solution would be to rebuild the multi-architecture image with the desired tag. But, one drawback of rebuilding the image is that it publishes an image that hasn’t been tested

After a lot of googling, I discovered that I could use the docker manifest command to ‘add’ the desired tag to the image. As I described in Part 2 - Building multi-architecture Docker images for Intel and ARM, a manifest for a multi-architecture image is a JSON object that points to a collection of architecture-specific images. For example:

% docker manifest inspect microservicesio/plantuml:test-build-11c53...
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1994,
         "digest": "sha256:...",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1994,
         "digest": "sha256:28899af3cb9a1756149adef730ba0596d4ea334727d5a2f47d26746c066f20b3",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}

This manifest, for example, specifies that the Arm-specific image is microservicesio/plantuml@sha256:28899af3cb9a1756149adef730ba0596d4ea334727d5a2f47d26746c066f20b3.

Adding the desired tag to an image is simply a matter of using docker manifest create to create a new manifest that references the previously created images and docker manifest push to push the manifest to Docker Hub.

Using docker buildx imagetools create to tag the image

I then stumbled across a simpler way to create and push a manifest using the docker buildx imagetools create command. The docker buildx imagetools provides a set of commands for working on images in a registry. For example, this command creates the specified multi-architecture image in Docker Hub from the previously pushed images.

docker buildx imagetools create -t microservicesio/plantuml:BUILD-999 \
  microservicesio/plantuml@sha256:28899af3cb9a1756149adef730ba0596d4ea334727d5a2f47d26746c066f20b3 ...

Since it’s manipulating JSON metadata, the command is extremely fast.

The deploy-artifacts.sh script invokes docker buildx imagetools create with the source images obtained by using jq. The jq commands extract the digests from the JSON manifest and turns them into image references:

SOURCES=$(docker manifest inspect docker.io/microservicesio/plantuml:${SRC_TAG} | \
   jq -r '.manifests[].digest  | sub("^"; "docker.io/microservicesio/plantuml@")')

docker buildx imagetools create -t ${TARGET_IMAGE} $SOURCES

Since none of these commands use the Docker Daemon, the deploy job does not need setup_remote_docker.

After making these changes, the master branch build published microservicesio/plantuml:BUILD-131 image. I then created the 0.3.0.RELEASE branch. Finally, I’m able to run PlantUML on my M1 MacBook!

Viewing the changes

To see the changes I made to the project, take a look at this Github commit.

Next steps

Configuring the CircleCI pipeline to publish a multi-architecture Docker image was great learning experience. It also enabled me to use PlantUML on my M1 Macbook! The next step is to apply the lessons I learned here to the Eventuate projects.