Multi-Arch Images with Docker Buildx and QEMU

Overview

Recently we have been planning to move to AWS Gravitron Instances, and if you look at the price and performace benchmarks compared to other instance types, Gravitron CPUs are much faster and cheaper, and this is good enough to draw attention and everyone would like to harness the power. But as the mythologies over the years have been telling stories about legends, holding a greater power requires one to go trough series of quests, and one such is migrating your workloads from intel/amd64 based CPUs to ARM64( that’s what Gravitron is all about).

If you want to be that legendary platform engineer who migrated workload to Gravitron Instances and saved thousands of bucks, or may be get them as raise in your pay (I’d choose the latter). You have come to right place.

The Challenge:

Now that we have talk about the quest it is, lets talk about what exactly are the challenges.
1. Docker only supports building for platform you are working on right now. Traditionally if you want to build an image for arm64 architecture, you’d need to get a raspberry pi

2. Even though you have access multiple devices, push and tag them separately or you’d need to combine those images and push to the registry in order to have multiple architectures supported with a common tag.

3. All the frustration you get into when managing it all separately, or may be write a custom script.

QEMU:

If this word is new for you, then probably you were not aware of the fact that you can emulate multiple architectures on a single host. Yes, you read it right, check it out, its been there since long,

I used it to emulate network appliances and cisco routers in GNS3 four years ago.

As the Wikipedia says, QEMU is a free and open-source hypervisor. It emulates the machine’s processor through dynamic binary translation and provides a set of different hardware and device models for the machine, enabling it to run a variety of guest operating systems.

Let’s first check what platforms your docker driver supports before installing qemu.

docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS  PLATFORMS
default * docker                  
 default default         running linux/amd64, linux/386

Assuming that you are on an ubuntu based system, lets now install qemu.

sudo apt-get install -y qemu qemu-user-static

qemu-user-static is to enable an execution of different multi-architecture containers

Now check supported architectures by running docker buildx ls again

docker buildx ls
NAME/NODE    DRIVER/ENDPOINT             STATUS  PLATFORMS
default      docker                              
  default    default                     running linux/amd64, linux/386, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/arm/v7, linux/arm/v6

Now that we have installed and added support for arm64 architecture, we can now intiate builder, as we can’t use docker driver directly to build for multiple platforms simultaneously, though you can build for single platform if you want, but that’s not our goal.

docker buildx create --use --name mybuilder
docker buildx ls
NAME/NODE    DRIVER/ENDPOINT             STATUS  PLATFORMS
mybuilder *  docker-container                    
  mybuilder0 unix:///var/run/docker.sock running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default      docker                              
  default    default                     running linux/amd64, linux/386, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/arm/v7, linux/arm/v6

You can see that the builder is now listed.

Now lets build a docker image for linux/amd64 and linux/arm64 architectures

I am using spring-boot-actuator-demo repo to test it out,

docker buildx build \
-t drpdishant/multiarch-demo \
--platform linux/amd64,linux/arm64 \
--push  .

This will build and push images for both platforms to dockerhub with single tag, and on the target host docker will automatically detect the platfomr all pull the corresponding image if available.

We’ll pull and inspect the image by explicitly defining the platform for now.

linux/amd64:

docker pull drpdishant/multiarch-demo --platform linux/amd64
docker inspect drpdishant/multiarch-demo | jq .[].Architecture
#Output:
"amd64"
docker run --rm --platform linux/amd64 drpdishant/multiarch-demo uname -m
#Output:
x86_64

linux/arm64:

docker pull drpdishant/multiarch-demo --platform linux/arm64
docker inspect drpdishant/multiarch-demo | jq .[].Architecture
#Output:
"arm64"
docker run --rm --platform linux/arm64 drpdishant/multiarch-demo uname -m
#Output:
aarch64

Lets check that our image is actually working or not. I am running on my machine with QEMU installed hence specifying the platform, you may check it out on actual AWS Gravitron Instance, without specifying platform, it will automatically pull and run arm64 image

docker run — rm -it -p 8080:8080 — platform linux/arm64 drpdishant/multiarch-demo

Now in a new terminal run:

curl -sL localhost:8080/actuator/health | jq .

It should print health status as below.

{
  "status": "UP",
  "components": {
    "custom": {
      "status": "UP",
      "details": {
        "app": "Alive and Kicking",
        "error": "Nothing! I'm good."
      }
    },
    "diskSpace": {
      "status": "UP",
      "details": {
        "total": 510405902336,
        "free": 434132635648,
        "threshold": 10485760
      }
    },
    "ping": {
      "status": "UP"
    }
  }
}

Conclusion: Now you know you can emulate multiple architectures with QEMU and build docker images for multiple platforms, you are now one step closer to harness the power of ARM64 and Gravitron.

References:

Leverage multi-CPU architecture support

GitHub - multiarch/qemu-user-static: `/usr/bin/qemu-*-static`