Forcing specific deployments pods to deploy on master

Question

I have cluster with three master nodes. Because master nodes have a NoSchedule taint by default, I added tolerations to my deployments. Kubernetes version is 1.5:

Deployment.yaml:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
  labels:
    kubernetes.io/cluster-service: "true"
spec:
  replicas: 3
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-lb
        name: nginx-ingress-lb
      annotations:
        scheduler.alpha.kubernetes.io/tolerations: |
          [
            {
              "key": "dedicated",
              "operator": "Equal",
              "value": "master",
              "effect": "NoSchedule"
            }
          ]
    spec:
    […]

Even after adding the tolerations the pods are deploying on the worker nodes only. I want to deploy these only on Master. Any help is appreciated. thanks

Answer 1

Tolerations do not guarantee that a pod will be scheduled an a node with that taint. It just means that it can tolerate such taints and schedule on those nodes. If you specifically want to deploy your pod on a specific node, use labels. Example, give your master node a label say dedicated=master and set nodeSelector for your pod to look for this label.

to add the label to you node:

kubectl label nodes name_of_your_node dedicated=master

Kubernetes 1.6 and above syntax

nodeSelector for your pod:

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
  labels:
    kubernetes.io/cluster-service: "true"
spec:
  replicas: 3
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-lb
        name: nginx-ingress-lb
      annotations:
    spec:
      nodeSelector:
        dedicated: master
      tolerations:
      - key: dedicated
        operator: Equal
        value: master
        effect: NoSchedule
    […]

You can also use affinity under spec:

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        matchExpressions:
        - key: dedicated
          operator: Equal
          values: ["master"]

Pre 1.6 syntax

nodeSelector for your pod:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
  labels:
    kubernetes.io/cluster-service: "true"
spec:
  replicas: 3
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-lb
        name: nginx-ingress-lb
      annotations:
        scheduler.alpha.kubernetes.io/tolerations: |
          [
            {
              "key": "dedicated",
              "operator": "Equal",
              "value": "master",
              "effect": "NoSchedule"
            }
          ]
    spec:
      nodeSelector:
        dedicated: master
    […]

Annotations instead of nodeSelector:

scheduler.alpha.kubernetes.io/affinity: >
  {
    "nodeAffinity": {
      "requiredDuringSchedulingIgnoredDuringExecution": {
        "nodeSelectorTerms": [
          {
            "matchExpressions": [
              {
                "key": "dedicated",
                "operator": "Equal",
                "values": ["master"]
              }
            ]
          }
        ]
      }
    }
  }

NoSchedule does not stop already scheduled pods. Check https://kubernetes.io/docs/user-guide/node-selection/ for more info.