Schedule pod on specific node without modifying PodSpec

Question

As a k8s cluster administrator, I want to specify on which nodes (using labels) pods will be scheduled, but without modifying any PodSpec section.

So, nodeSelector, affinity and taints can't be options. Is there any other solution ?

PS: the reason I can't modify the PodSpec is that deployed applications are available as Helm charts and I don't have hand on those files. Moreover, if I change the PodSpec, it will be lost on next release upgrade.

Answer 1

You can use the PodNodeSelector admission controller for this:

This admission controller has the following behavior:

1. If the Namespace has an annotation with a key scheduler.kubernetes.io/nodeSelector, use its value as the node selector.
2. If the namespace lacks such an annotation, use the clusterDefaultNodeSelector defined in the PodNodeSelector plugin configuration file as the node selector.
3. Evaluate the pod’s node selector against the namespace node selector for conflicts. Conflicts result in rejection.
4. Evaluate the pod’s node selector against the namespace-specific whitelist defined the plugin configuration file. Conflicts result in rejection.

First of all you will need to enable this admission controller. The way to enable it depends on your environment, but it's done via the parameter kube-apiserver --enable-admission-plugins=PodNodeSelector.

Then create a namespace and annotate it with whatever node label you want all Pods in that namespace to have:

kubectl create ns node-selector-test
kubectl annotate ns node-selector-test \
    scheduler.alpha.kubernetes.io/node-selector=mynodelabel=mynodelabelvalue

To test it you could do something like this:

kubectl run busybox \
    -n node-selector-test -it --restart=Never --attach=false --image=busybox

kubectl get pod busybox -n node-selector-test -o yaml

It should output something like this:

apiVersion: v1
kind: Pod 
metadata:
  name: busybox
  ....
spec:
  ...
  nodeSelector:
    mynodelabel: mynodelabelvalue

Now, unless that label exists on some nodes, this Pod will never be scheduled, so put this label on a node to see it scheduled:

kubectl label node myfavoritenode mynodelabel=mynodelabelvalue