K8S中的Job和CronJob

Job

Job负责批量处理短暂的一次性任务 (short lived one-off tasks)，即仅执行一次的任务，它保证批处理任务的一个或多个Pod成功结束。

Kubernetes支持以下几种Job：

• 非并行Job：通常创建一个Pod直至其成功结束
• 固定结束次数的Job：设置.spec.completions，创建多个Pod，直到.spec.completions个Pod成功结束
• 带有工作队列的并行Job：设置.spec.Parallelism但不设置.spec.completions，当所有Pod结束并且至少一个成功时，Job就认为是成功

根据.spec.completions和.spec.Parallelism的设置，可以将Job划分为以下几种pattern：

Job类型	使用示例	行为	completions	Parallelism
一次性Job	数据库迁移	创建一个Pod直至其成功结束	1	1
固定结束次数的Job	处理工作队列的Pod	依次创建一个Pod运行直至completions个成功结束	2+	1
固定结束次数的并行Job	多个Pod同时处理工作队列	依次创建多个Pod运行直至completions个成功结束	2+	2+
并行Job	多个Pod同时处理工作队列	创建一个或多个Pod直至有一个成功结束	1	2+

Job Controller

Job Controller负责根据Job Spec创建Pod，并持续监控Pod的状态，直至其成功结束。如果失败，则根据restartPolicy（只支持OnFailure和Never，不支持Always）决定是否创建新的Pod再次重试任务。

Job Spec格式

• spec.template格式同Pod
• RestartPolicy仅支持Never或OnFailure
• 单个Pod时，默认Pod成功运行后Job即结束
• .spec.completions标志Job结束需要成功运行的Pod个数，默认为1
• .spec.parallelism标志并行运行的Pod的个数，默认为1
• spec.activeDeadlineSeconds标志失败Pod的重试最大时间，超过这个时间不会继续重试

一个简单的例子：

[root@k8s-master mnt]# cat job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: pi
spec:
  template:
    metadata:
      name: pi
    spec:
      containers:
      - name: pi
        image: perl
        command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
[root@k8s-master mnt]#

[root@k8s-master mnt]# kubectl create -f  job.yaml
job.batch/pi created
[root@k8s-master mnt]# kubectl get pod
NAME                      READY   STATUS              RESTARTS   AGE
deamonset-example-vdzjp   1/1     Running             0          19m
deamonset-example-xxt2z   1/1     Running             0          20m
pi-7s4vx                  0/1     ContainerCreating   0          5s
[root@k8s-master mnt]# kubectl get pod
NAME                      READY   STATUS              RESTARTS   AGE
deamonset-example-vdzjp   1/1     Running             0          19m
deamonset-example-xxt2z   1/1     Running             0          20m
pi-7s4vx                  0/1     ContainerCreating   0          7s
[root@k8s-master mnt]# kubectl describe pod pi-7s4vx
Name:           pi-7s4vx
Namespace:      default
Priority:       0
Node:           k8s-node02/192.168.180.136
Start Time:     Mon, 23 Dec 2019 20:21:25 +0800
Labels:         controller-uid=51af6eb5-9166-49eb-9328-8f5c6bcfe18f
                job-name=pi
Annotations:    <none>
Status:         Pending
IP:
IPs:            <none>
Controlled By:  Job/pi
Containers:
  pi:
    Container ID:
    Image:         perl
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Command:
      perl
      -Mbignum=bpi
      -wle
      print bpi(2000)
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-6wcrh (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-6wcrh:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-6wcrh
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age        From                 Message
  ----    ------     ----       ----                 -------
  Normal  Scheduled  <unknown>  default-scheduler    Successfully assigned default/pi-7s4vx to k8s-node02
  Normal  Pulling    19s        kubelet, k8s-node02  Pulling image "perl"
[root@k8s-master mnt]# kubectl get job
NAME   COMPLETIONS   DURATION   AGE
pi     0/1           26s        26s
[root@k8s-master mnt]# kubectl get job
NAME   COMPLETIONS   DURATION   AGE
pi     0/1           31s        31s
[root@k8s-master mnt]# kubectl describe pod pi-7s4vx
Name:           pi-7s4vx
Namespace:      default
Priority:       0
Node:           k8s-node02/192.168.180.136
Start Time:     Mon, 23 Dec 2019 20:21:25 +0800
Labels:         controller-uid=51af6eb5-9166-49eb-9328-8f5c6bcfe18f
                job-name=pi
Annotations:    <none>
Status:         Pending
IP:
IPs:            <none>
Controlled By:  Job/pi
Containers:
  pi:
    Container ID:
    Image:         perl
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Command:
      perl
      -Mbignum=bpi
      -wle
      print bpi(2000)
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-6wcrh (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-6wcrh:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-6wcrh
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age        From                 Message
  ----    ------     ----       ----                 -------
  Normal  Scheduled  <unknown>  default-scheduler    Successfully assigned default/pi-7s4vx to k8s-node02
  Normal  Pulling    2m11s      kubelet, k8s-node02  Pulling image "perl"

[root@k8s-master mnt]# kubectl get pod
NAME                      READY   STATUS      RESTARTS   AGE
deamonset-example-vdzjp   1/1     Running     0          28m
deamonset-example-xxt2z   1/1     Running     0          29m
pi-7s4vx                  0/1     Completed   0          9m7s
[root@k8s-master mnt]# kubectl get pod -o wide
NAME                      READY   STATUS      RESTARTS   AGE     IP            NODE         NOMINATED NODE   READINESS GATES
deamonset-example-vdzjp   1/1     Running     0          28m     10.244.2.22   k8s-node01   <none>           <none>
deamonset-example-xxt2z   1/1     Running     0          29m     10.244.1.23   k8s-node02   <none>           <none>
pi-7s4vx                  0/1     Completed   0          9m15s   10.244.1.24   k8s-node02   <none>           <none>
[root@k8s-master mnt]# kubectl logs pi-7s4vx
3.141592653589793238462643383279502884197169399375105820974944592307816678316527120190914564856692346034

固定结束次数的Job示例

apiVersion: batch/v1
kind: Job
metadata:
  name: busybox
spec:
  completions: 3
  template:
    metadata:
      name: busybox
    spec:
      containers:
      - name: busybox
        image: busybox
        command: ["echo", "hello"]
      restartPolicy: Never

CronJob

CronJob即定时任务，就类似于Linux系统的crontab，在指定的时间周期运行指定的任务。在Kubernetes 1.5，使用CronJob需要开启batch/v2alpha1 API，即–runtime-config=batch/v2alpha1。

CronJob Spec

• .spec.schedule指定任务运行周期，格式同Cron
• .spec.jobTemplate指定需要运行的任务，格式同Job
• .spec.startingDeadlineSeconds指定任务开始的截止期限
• .spec.concurrencyPolicy指定任务的并发策略，支持Allow、Forbid和Replace三个选项

[root@k8s-master mnt]# cat cronjob.yaml
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: busybox
            args:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
          restartPolicy: OnFailure
[root@k8s-master mnt]#

[root@k8s-master mnt]# vim cronjob.yaml
[root@k8s-master mnt]# kubectl create -f cronjob.yaml
cronjob.batch/hello created
[root@k8s-master mnt]# kubectl get cronjob
NAME    SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
hello   */1 * * * *   False     0        <none>          16s
[root@k8s-master mnt]# kubectl get job
NAME               COMPLETIONS   DURATION   AGE
hello-1577105220   0/1           2s         2s
pi                 1/1           5m23s      25m
[root@k8s-master mnt]# kubectl get pod
NAME                      READY   STATUS      RESTARTS   AGE
deamonset-example-vdzjp   1/1     Running     0          45m
deamonset-example-xxt2z   1/1     Running     0          47m
hello-1577105220-9vjjn    0/1     Completed   0          50s
pi-7s4vx                  0/1     Completed   0          26m
[root@k8s-master mnt]# kubectl logs pod hello-1577105220-9vjjn
Error from server (NotFound): pods "pod" not found
[root@k8s-master mnt]# kubectl logs hello-1577105220-9vjjn
Mon Dec 23 12:47:14 UTC 2019
Hello from the Kubernetes cluster
[root@k8s-master mnt]# kubectl get job -w
NAME               COMPLETIONS   DURATION   AGE
hello-1577105220   1/1           9s         2m4s
hello-1577105280   1/1           8s         64s
hello-1577105340   0/1           4s         4s
pi                 1/1           5m23s      27m
hello-1577105340   1/1           6s         6s
hello-1577105400   0/1                      0s
hello-1577105400   0/1           0s         0s
hello-1577105400   1/1           5s         5s
^Z
[1]+  已停止               kubectl get job -w
[root@k8s-master mnt]# kubectl delete -f cronjob.yaml
cronjob.batch "hello" deleted
[root@k8s-master mnt]# kubectl delete -f job.yaml
job.batch "pi" deleted
[root@k8s-master mnt]#