Node Scheduling

Node Affinity, Pod Affinity/Anti-Affinity, Toleration/Taint
강의자료 PDF
Scheduling with Affinity and Toleration Taint for Kubernetes.


1. Node Affinity

Scheduling of Node Affinity for Kubernetes.

1-1) Node Labeling

kubectl label nodes k8s-node1 kr=az-1
kubectl label nodes k8s-node2 us=az-1

1-2) MatchExpressions

apiVersion: v1
kind: Pod
metadata:
 name: pod-match-expressions1
spec:
 affinity:
  nodeAffinity:
   requiredDuringSchedulingIgnoredDuringExecution:   
    nodeSelectorTerms:
    - matchExpressions:
      -  {key: kr, operator: Exists}
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

1-3) Required

apiVersion: v1
kind: Pod
metadata:
 name: pod-required
spec:
 affinity:
  nodeAffinity:
   requiredDuringSchedulingIgnoredDuringExecution:
    nodeSelectorTerms:
    - matchExpressions:
      - {key: ch, operator: Exists}
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

1-4) Preferred

apiVersion: v1
kind: Pod
metadata:
 name: pod-preferred
spec:
 affinity:
  nodeAffinity:
   preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 1
      preference:
       matchExpressions:
       - {key: ch, operator: Exists}
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

2. Pod Affinity / Anti-Affinity

Scheduling of Pod Affinity for Kubernetes.

2-1) Node Labeling

kubectl label nodes k8s-node1 a-team=1
kubectl label nodes k8s-node2 a-team=2

2-2) Web1 Pod

apiVersion: v1
kind: Pod
metadata:
 name: web1
 labels:
  type: web1
spec:
 nodeSelector:
  a-team: '1'
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

2-3) Web1 Affinity Pod

apiVersion: v1
kind: Pod
metadata:
 name: server1
spec:
 affinity:
  podAffinity:
   requiredDuringSchedulingIgnoredDuringExecution:   
   - topologyKey: a-team
     labelSelector:
      matchExpressions:
      -  {key: type, operator: In, values: [web1]}
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

2-4) Web2 Affinity Pod

apiVersion: v1
kind: Pod
metadata:
 name: server2
spec:
 affinity:
  podAffinity:
   requiredDuringSchedulingIgnoredDuringExecution:   
   - topologyKey: a-team
     labelSelector:
      matchExpressions:
      -  {key: type, operator: In, values: [web2]}
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

2-5) Web2 Pod

apiVersion: v1
kind: Pod
metadata:
  name: web2
  labels:
     type: web2
spec:
  nodeSelector:
    a-team: '2'
  containers:
  - name: container
    image: kubetm/app
  terminationGracePeriodSeconds: 0

Scheduling of Pod Anti-Affinity for Kubernetes.

2-6) Master Pod

apiVersion: v1
kind: Pod
metadata:
  name: master
  labels:
     type: master
spec:
  nodeSelector:
    a-team: '1'
  containers:
  - name: container
    image: kubetm/app
  terminationGracePeriodSeconds: 0

2-7) Master Anti-Affinity Pod

apiVersion: v1
kind: Pod
metadata:
 name: slave
spec:
 affinity:
  podAntiAffinity:
   requiredDuringSchedulingIgnoredDuringExecution:   
   - topologyKey: a-team
     labelSelector:
      matchExpressions:
      -  {key: type, operator: In, values: [master]}
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

3. Taint / Toleration

Scheduling of Taint and Toleration for Kubernetes.

3-1) Node Labeling

kubectl label nodes k8s-node1 gpu=no1

3-2) Node1 - Taint

kubectl taint nodes k8s-node1 hw=gpu:NoSchedule

3-3) Pod with Toleration

apiVersion: v1
kind: Pod
metadata:
 name: pod-with-toleration
spec:
 nodeSelector:
  gpu: no1
 tolerations:
 - effect: NoSchedule
   key: hw
   operator: Equal
   value: gpu
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

3-4) Pod without Toleration

apiVersion: v1
kind: Pod
metadata:
 name: pod-without-toleration
spec:
 nodeSelector:
  gpu: no1
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

3-5) Pod1 with NoExecute

apiVersion: v1
kind: Pod
metadata:
 name: pod1-with-no-execute
spec:
 tolerations:
 - effect: NoExecute
   key: out-of-disk
   operator: Exists
   tolerationSeconds: 30
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

3-6) Pod2 with NoExecute

apiVersion: v1
kind: Pod
metadata:
 name: pod2-without-no-execute
spec:
 containers:
 - name: container
   image: kubetm/app
 terminationGracePeriodSeconds: 0

3-7) Node2 - Taint

kubectl taint nodes k8s-node1 hw=gpu:NoSchedule-
kubectl taint nodes k8s-node2 out-of-disk=True:NoExecute


Sample Yaml

NodeName

apiVersion: v1
kind: Pod
metadata:
 name: node-name
spec:
 nodeName: k8s-node1
 containers:
 - name: container
   image: kubetm/app

NodeSelector

apiVersion: v1
kind: Pod
metadata:
 name: node-selector
spec:
 nodeSelector: 
  key1:value1
 containers:
 - name: container
   image: kubetm/app

kubectl

Label

# Node에 라벨 추가
kubectl label nodes k8s-node1 key1=value1
# Node에 라벨 삭제
kubectl label nodes k8s-node1 key1=value1-


Tips

NodeAffinity MatchExpressions Operator

Scheduling MatchExpressions for Kubernetes.


Referenece

Kubernetes