How do it create a shutdown sequence for containers in a pod?

Question

when a SIGTERM is received from k8s I want my sidecar to die only after the main container is finished. How do I create a dependency chain for shutdown routine?

I can use a preStop and wait for say 120 secs, but it is going to a constant 120 secs for the pods to be cleaned up.

I am wondering if there is a way for my side car to check if my main container was killed. Or is it possible for the main container to signal my side car when it has finished its clean up (through k8s instead of a code change in my main container.

Answer 1

In your preStop hook, check if your main container is still running (instead of just waiting a fixed amount of time.

If the main container has an existing healthprobe (for example an http endpoint) you can call that from within the preStop hook.

Another option is to edit the entrypoint script in your main container to include a trap command that creates a file on shutdown. The preStop hook of your sidecar to wait for this file be present.

trap "touch /lifecycle/main-pod-terminated" SIGTERM

If using the preStop hook, just keep the grace period countdown in mind and increase the default (30 seconds) if required by setting an appropriate value of terminationGracePeriodSeconds

Pod's termination grace period countdown begins before the preStop hook is executed, so regardless of the outcome of the handler, the container will eventually terminate within the Pod's termination grace period