This is the fifth article about my adventures trying to use my Apple M1 MacBook for development.
In the previous article, I described how to configure a CircleCI-based CI/CD pipeline to test a Docker image on both Intel and ARM platforms.
Currently, however, the pipeline doesn’t do anything with the image once the Arm-platform tests pass.
It leaves the image in Docker Hub with the tag test-build-${CIRCLE_SHA1?}
.
In this article, I describe how to configure the pipeline to add a ‘proper’ tag (e.g. BUILD-*
, x.y.z.RELEASE
, etc) to the image without rebuilding it.
The other articles in this series are:
deploy
jobThe pipeline’s build
job is still running the original deploy-artifacts.sh
script which pushes the single architecture image to Docker Hub before testing the multi-architecture image on Arm.
This no longer makes sense and so the first change is to define deploy
job, which only runs deploy-artifacts.sh
if the test-arm64
job succeeds:
...
deploy:
docker:
- image: cimg/base:stable
working_directory: ~/plantuml
steps:
- checkout
- setup_remote_docker:
version: 20.10.11
- run:
command: ./deploy-artifacts.sh
workflows:
version: 2.1
build-test-and-deploy:
jobs:
- build
- test-arm64:
requires:
- build
- deploy:
requires:
- test-arm64
The deploy
job executes once the test-arm64
jobs succeeds.
Let’s now look at how to publish the multi-architecture with the correct tag.
The original deploy-artifacts.sh
script simply pushed the locally built image to Docker Hub that had a tag corresponding to the Git branch.
For example, a master
branch build would push a BUILD-${CIRCLE_BUILD_NUM?}
tag, a x.y.z.RELEASE
branch build would push a x.y.z.RELEASE
tag.
What’s different about building a multi-architecture image is that it’s already been pushed to Docker Hub with a test-build-${CIRCLE_SHA1?}
tag.
I naively thought that I could assign the correct tag by simply executing the following sequence of commands:
docker pull
docker tag
to add a desired tagdocker push
These commands successfully publish an appropriately tagged image. However, that image is an Intel-only image! Yet another reminder that a Docker daemon only supports a single architecture. Pulling a multi-architecture image only downloads an image for that architecture.
One solution would be to rebuild the multi-architecture image with the desired tag. But, one drawback of rebuilding the image is that it publishes an image that hasn’t been tested
After a lot of googling, I discovered that I could use the docker manifest
command to ‘add’ the desired tag to the image.
As I described in Part 2 - Building multi-architecture Docker images for Intel and ARM, a manifest for a multi-architecture image is a JSON object that points to a collection of architecture-specific images.
For example:
% docker manifest inspect microservicesio/plantuml:test-build-11c53...
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 1994,
"digest": "sha256:...",
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 1994,
"digest": "sha256:28899af3cb9a1756149adef730ba0596d4ea334727d5a2f47d26746c066f20b3",
"platform": {
"architecture": "arm64",
"os": "linux"
}
}
]
}
This manifest, for example, specifies that the Arm-specific image is microservicesio/plantuml@sha256:28899af3cb9a1756149adef730ba0596d4ea334727d5a2f47d26746c066f20b3
.
Adding the desired tag to an image is simply a matter of using docker manifest create
to create a new manifest that references the previously created images and docker manifest push
to push the manifest to Docker Hub.
docker buildx imagetools create
to tag the imageI then stumbled across a simpler way to create and push a manifest using the docker buildx imagetools create
command.
The docker buildx imagetools
provides a set of commands for working on images in a registry.
For example, this command creates the specified multi-architecture image in Docker Hub from the previously pushed images.
docker buildx imagetools create -t microservicesio/plantuml:BUILD-999 \
microservicesio/plantuml@sha256:28899af3cb9a1756149adef730ba0596d4ea334727d5a2f47d26746c066f20b3 ...
Since it’s manipulating JSON metadata, the command is extremely fast.
The deploy-artifacts.sh
script invokes docker buildx imagetools create
with the source images obtained by using jq
.
The jq
commands extract the digests from the JSON manifest and turns them into image references:
SOURCES=$(docker manifest inspect docker.io/microservicesio/plantuml:${SRC_TAG} | \
jq -r '.manifests[].digest | sub("^"; "docker.io/microservicesio/plantuml@")')
docker buildx imagetools create -t ${TARGET_IMAGE} $SOURCES
Since none of these commands use the Docker Daemon, the deploy
job does not need setup_remote_docker
.
After making these changes, the master branch build published microservicesio/plantuml:BUILD-131
image.
I then created the 0.3.0.RELEASE
branch.
Finally, I’m able to run PlantUML on my M1 MacBook!
To see the changes I made to the project, take a look at this Github commit.
Configuring the CircleCI pipeline to publish a multi-architecture Docker image was great learning experience. It also enabled me to use PlantUML on my M1 Macbook! The next step is to apply the lessons I learned here to the Eventuate projects.
Microservices.io is brought to you by Chris Richardson. Experienced software architect, author of POJOs in Action, the creator of the original CloudFoundry.com, and the author of Microservices patterns.
Chris helps clients around the world adopt the microservice architecture through consulting engagements, and training workshops.
Take a look at my Manning LiveProject that teaches you how to develop a service template and microservice chassis.
My virtual bootcamp, distributed data patterns in a microservice architecture, is now open for enrollment!
It covers the key distributed data management patterns including Saga, API Composition, and CQRS.
It consists of video lectures, code labs, and a weekly ask-me-anything video conference repeated in multiple timezones.
The regular price is $395/person but use coupon JUNVCEJE to sign up for $195 (valid until February 1st, 2023). There are deeper discounts for buying multiple seats.
Chris offers numerous resources for learning the microservice architecture.
Chris teaches comprehensive workshops, training classes and bootcamps for executives, architects and developers to help your organization use microservices effectively.
Avoid the pitfalls of adopting microservices and learn essential topics, such as service decomposition and design and how to refactor a monolith to microservices.
Delivered in-person and remotely.
Want to see an example? Check out Chris Richardson's example applications. See code
Engage Chris to create a microservices adoption roadmap and help you define your microservice architecture,
Use the Eventuate.io platform to tackle distributed data management challenges in your microservices architecture.
Eventuate is Chris's latest startup. It makes it easy to use the Saga pattern to manage transactions and the CQRS pattern to implement queries.
Engage Chris to conduct an architectural assessment.
Note: tagging is work-in-process
anti-patterns · application api · application architecture · architecting · architecture documentation · dark energy and dark matter · deployment · development · devops · docker · implementing commands · implementing queries · inter-service communication · loose coupling · microservice architecture · microservice chassis · microservices adoption · microservicesio updates · multi-architecture docker images · observability · pattern · refactoring to microservices · resilience · sagas · security · service api · service collaboration · service design · service discovery · service granularity · service template · software delivery metrics · success triangle · team topologies · transaction management · transactional messaging