Building a multi-architecture Docker image for Apache Kafka

multi-architecture docker images   docker  

---

In the previous post, I described how I configured the CircleCI pipeline for the Eventuate Common project to build test and publish MySQL8 and Zookeeper Docker images In this post, I cover how I changed the eventuate-messaging-kafka project to publish a multi-architecture Apache Kafka image needed by the Eventuate Tram Customers and Orders example.

The other articles in this series are:

• Part 1 - My Apple M1 MacBook: lots of cores, memory and crashing containers
• Part 2 - Building multi-architecture Docker images for Intel and ARM
• Part 3 - Configuring a CircleCI-based pipeline to build multi-architecture Docker images
• Part 4 - Testing an Intel and Arm multi-architecture Docker image on CircleCI
• Part 5 - Configuring CircleCI to publish a multi-architecture Docker image
• Part 6 - Developing the Eventuate Common library on an M1/Arm MacBook
• Part 7 - Configuring CircleCI to build, test and publish multi-architecture images for Eventuate Common
• Part 9 - Publishing multi-architecture base images for services
• Part 10 - Publishing a multi-architecture Docker image for the Eventuate CDC service
• Part 11 - The Eventuate Tram Customers and Orders example now runs on Arm/M1 MacBook!!

Why build a multi-architecture Apache Kafka image?

Eventuate Tram Customers and Orders, like the other Eventuate examples, uses the Confluent Platform Kafka image. Surprisingly, this is an Intel-only image, so I needed to build my own a multi-architecture image. But which Eventuate project should do this?

Changing the eventuate-messaging-kafka project to publish an Apache Kafka image

I decided that it made sense for the eventuate-messaging-kafka project to build the and publish the Apache Kafka Docker image . It’s a Gradle project that defines the Apache Kafka implementations of the Eventuate consumer and producers. Its tests need to run Apache Kafka and they currently use the Confluent Platform Kafka image. By changing this project to build the Apache Kafka image, I can reuse those tests to validate the image.

Changing the project’s CircleCI pipeline

I modified the project’s deployment pipeline to follow that the same four step pattern that I used for Eventuate Common:

1. Build a multi-architecture Apache Kafka image locally (using a registry container), and run the Java tests on both Intel and Arm.
2. Build a multi-architecture Apache Kafka image and push it to Docker Hub with a test-build-* tag
3. Test the multi-architecture test-build-* image on both Intel and Arm.
4. Create a manifest in Docker Hub with the desired tag

Viewing the changes

To see the changes I made to the project, take a look at this Github commit and a bug fix commit

In the next article, I’ll describe the changes that I made to the eventuate-examples-docker-images project, which publishes the base Docker images for the services in the example applications.