Compute Engine Deploy Container

Question

I am using golang to programmatically create and destroy one-off Compute Engine instances using the Compute Engine API.

I can create an instance just fine, but what I'm really having trouble with is launching a container on startup.

You can do it from the Console UI:

But as far as I can tell it's extremely hard to do it programmatically, especially with Container Optimized OS as the base image. I tried doing a startup script that does a docker pull us-central1-docker.pkg.dev/project/repo/image:tag but it fails because you need to do gcloud auth configure-docker us-central1-docker.pkg.dev first for that to work and COOS doesn't have gcloud nor a package manager to get it.

All my workarounds seem hacky:

• Manually create a VM template that has the desired container and create instances of the template
• Put container in external registry like docker hub (not acceptable)
• Use Ubuntu instead of COOS with a package manager so I can programmatically install gcloud, docker, and the container on startup
• Use COOS to pull down an image from dockerhub containing gcloud, then do some sort of docker-in-docker mount to pull it down

Am I missing something or is it just really cumbersome to deploy a container to a compute engine instance without using gcloud or the Console UI?

Answer 1

To have a Compute Engine start a container when the Compute Engine starts, one has to define meta data for the description of the container. When the COOS starts, it appears to run an application called konlet which can be found here:

https://github.com/GoogleCloudPlatform/konlet

If we look at the documentation for this, it says:

The agent parses container declaration that is stored in VM instance metadata under gce-container-declaration key and starts the container with the declared configuration options.

Unfortunately, I haven't found any formal documentation for the structure of this metadata. While I couldn't find documentation, I did find two possible solutions:

1. Decipher the source code of konlet and break it apart to find out how the metadata maps to what is passed when the docker container is started

or

2. Create a Compute Engine by hand with the desired container definitions and then start the Compute Engine. SSH into the Compute Engine and then retrieve the current metadata. We can read about retrieving meta data here:

https://cloud.google.com/compute/docs/metadata/overview

Answer 2

It turns out, it's not too hard to pull down a container from Artifact Registry in Container Optimized OS:

• Run docker-credential-gcr configure-docker --registries [region]-docker.pkg.dev

See: https://cloud.google.com/container-optimized-os/docs/how-to/run-container-instance#accessing_private_images_in_or

So what you can do is put the above line along with docker pull [image] and docker run ... into a startup script. You can specify a startup script when creating an instance using the metadata field: https://cloud.google.com/compute/docs/instances/startup-scripts/linux#api

This seems the least hacky way of provisioning an instance with a container programmatically.