Tagging multi-platform images in ECR creates untagged manifests

Question

I've started using docker buildx to tag and push mutli-platform images to ECR. However, ECR appears to apply the tag to the parent manifest, and leaves each related manifest as untagged. ECR does appear to prevent deletion of the child manifests, but it makes managing cleanup of orphaned untagged images complicated.

Is there a way to tag these child manifests in some way?

For example, consider this push:

docker buildx build --platform "linux/amd64,linux/arm64" --tag 1234567890.dkr.ecr.eu-west-1.amazonaws.com/my-service/my-image:1.0 --push  . 

Inspecting the image:

 docker buildx imagetools inspect 1234567890.dkr.ecr.eu-west-1.amazonaws.com/my-service/my-image:1.0

Shows:

Name:      1234567890.dkr.ecr.eu-west-1.amazonaws.com/my-service/my-image:1.0
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:4221ad469d6a18abda617a0041fd7c87234ebb1a9f4ee952232a1287de73e12e
       
Manifests: 
    Name:      1234567890.dkr.ecr.eu-west-1.amazonaws.com/my-service/my-image:1.0@sha256:c1b0c04c84b025357052eb513427c8b22606445cbd2840d904613b56fa8283f3
    MediaType: application/vnd.docker.distribution.manifest.v2+json
    Platform:  linux/amd64
         
    Name:      1234567890.dkr.ecr.eu-west-1.amazonaws.com/my-service/my-image:1.0@sha256:828414cad2266836d9025e9a6af58d6bf3e6212e2095993070977909ee8aee4b
    MediaType: application/vnd.docker.distribution.manifest.v2+json
    Platform:  linux/arm64

However, ECR shows the 2 child images as untagged

Answer 1

I'm running into the same problem. So far my solution seems a little easier than some of the other suggestions, but I still don't like it.

After doing the initial:

docker buildx build --platform "linux/amd64,linux/arm64" --tag 1234567890.dkr.ecr.eu-west-1.amazonaws.com/my-service/my-image:1.0 --push  . 

I follow up with:

docker buildx build --platform "linux/amd64" --tag 1234567890.dkr.ecr.eu-west-1.amazonaws.com/my-service/my-image:1.0-amd --push  .
docker buildx build --platform "linux/arm64" --tag 1234567890.dkr.ecr.eu-west-1.amazonaws.com/my-service/my-image:1.0-arm --push  .

This gets me the parallel build speed of building multiple platforms at the same time, and gets me the images tagged in ECR. Thanks to having the build info cached it is pretty quick, it appears to just push the tags and that is it. In a test I just did the buildx time for the first command was 0.5 seconds. and the second one took 0.7 seconds.

That said, I'm not wild about this solution, and found this question while looking for a better one.

Answer 2

There are several ways to tag the image, but they all involve pushing the platform specific manifest with the desired tag. With docker, you can pull the image, retag it, and push it, but the downside is you'll have to pull every layer.

A much faster option is to only transfer the manifest json with registry API calls. You could do this with curl, but auth becomes complicated. There are several tools for working directly with registries, including Googles crane, RedHat's skopeo, and my own regclient. Regclient includes the regctl command which would implement this like:

image=1234567890.dkr.ecr.eu-west-1.amazonaws.com/my-service/my-image
tag=1.0
regctl image copy \
  ${image}@$(regctl image digest --platform linux/amd64 $image:$tag) \
  ${image}:${tag}-linux-amd64
regctl image copy \
  ${image}@$(regctl image digest --platform linux/arm64 $image:$tag) \
  ${image}:${tag}-linux-arm64

You could also script an automated fix to this, listing all tags in the registry, pulling the manifest list for the tags that don't already have the platform, and running the image copy to retag each platform's manifest. But it's probably easier and faster to script your buildx job to include something like regctl after buildx pushes the image.

Note if you use a cred helper for logging into ECR, regctl can use this with the local command. If want to run regctl as a container, and you are specifically using ecr-login, use the alpine version of the images since they include the helper binary.