How to terminate janusgraph container in case any exception is thrown

Question

I'm using janusgraph docker image - https://hub.docker.com/r/janusgraph/janusgraph In my kubernetes deployment to initialise the remote graph using groovy script mounted to docker-entrypoint-initdb.d

This works as expected but in case if the remote host is not ready the janusgraph container throws exception and is still in the running mode.

Because of this kubernetes will not attempt to restart the container again. Is there any way so that I can configure this janusgraph container to terminate in case of any exception

Answer 1

As @Gavin has mentioned you can use probes to check if containers are working. Liveness Probes is used to know when containers are failed. If a container is unresponsive - it can restart the container.

Readiness probes inform when the container is available for accepting traffic. The readiness probe is used to control which pods are used as the backends for a service. A pod is considered ready when all of its containers are ready. If a pod is not ready, it is removed from service Endpoints.

Kubernetes supports three mechanisms for implementing liveness and readiness probes:

1) making an HTTP request against a container This probes have additional fields that can be set on httpGet:

• host: Host name to connect to, defaults to the pod IP. You probably want to set "Host" in httpHeaders instead.
• scheme: Scheme to use for connecting to the host (HTTP or HTTPS). Defaults to HTTP.
• path: Path to access on the HTTP server. Defaults to /.
• httpHeaders: Custom headers to set in the request. HTTP allows repeated headers.
• port: Name or number of the port to access on the container. Number must be in the range 1 to 65535.

Read more: http-probes.

livenessProbe:
  httpGet:
    path: /healthz
    port: liveness-port

2) opening a TCP socket against a container

initialDelaySeconds: 15  
livenessProbe: ~  
periodSeconds: 20  
port: 8080  
tcpSocket: ~

3) running a command inside a container

livenessProbe:  
  exec:  
    command:  
    - sh  
    - /tmp/status_check.sh  
  initialDelaySeconds: 10  

If you will get status code different than 0 this will mean that probe failed. You can also add to probes additional params such as initialDelaySeconds: indicate number of seconds after the container has started before liveness or readiness probes are initiated. See: configuring-probes.

In every case add also restartPolicy: Never to your pods definition. By default is always.

Answer 2

A readinessProbe could be employed here with a command like janusgraph show-config or something similar which will exit with code -1

spec:
  containers:
  - name: liveness
    image: janusgraph/janusgraph:latest
    readinessProbe:
      exec:
        command:
        - janusgraph 
        - show-config

Kubernetes will terminate the pod if the readinessProbe fails. A livenessProbe could also be used here too, in case this pod needs to be terminated if the remote host ever becomes unavailable.

Consider enabling JanusGraph server metrics, which could then be used with Prometheus for additional monitoring or even with the livenessProbe itself.