9. 第八篇 kube-controller-manager安装及验证

文章转载自：https://mp.weixin.qq.com/s?__biz=MzI1MDgwNzQ1MQ==&mid=2247483826&idx=1&sn=88f0cef6866f386fb6d779dcecded534&chksm=e9fdd446de8a5d509f69f62b328dc910d0949d1431be58e2d04eba9f5b95d74651c028d6d215&cur_album_id=1341273083637989377&scene=189#wechat_redirect

kube-controller-manager（k8s控制器管理器）是一个守护进程，它通过kube-apiserver监视集群的共享状态（kube-apiserver收集或监视到的一些集群资源状态，供kube-controller-manager或其它客户端watch）, 控制器管理器并尝试将当前的状态向所定义的状态迁移（移动、靠近），它本身是有状态的，会修改集群状态信息，如果多个控制器管理器同时生效，则会有一致性问题，所以kube-controller-manager的高可用，只能是主备模式，而kubernetes集群是采用租赁锁实现leader选举，需要在启动参数中加入--leader-elect=true。

下载二进制文件https://dl.k8s.io/v1.17.0/kubernetes-server-linux-amd64.tar.gz，并把kube-controller-manager二进制文件拷贝到master节点的/data/k8s/bin/此目录

创建kubeconfig配置文件并分发

kube-controller-manager也是作为kube-apiserver的客户端，我们需要创建kubeconfig文件。

#!/bin/bash

cd /data/k8s/work
source /data/k8s/bin/env.sh

kubectl config set-cluster kubernetes \
  --certificate-authority=/data/k8s/work/ca.pem \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-credentials system:kube-controller-manager \
  --client-certificate=kube-controller-manager.pem \
  --client-key=kube-controller-manager-key.pem \
  --embed-certs=true \
  --kubeconfig=kube-controller-manager.kubeconfig
kubectl config set-context system:kube-controller-manager \
  --cluster=kubernetes \
  --user=system:kube-controller-manager \
  --kubeconfig=kube-controller-manager.kubeconfig
kubectl config use-context system:kube-controller-manager --kubeconfig=kube-controller-manager.kubeconfig

# 分发
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    scp kube-controller-manager.kubeconfig root@${node_ip}:/etc/kubernetes/
done

创建kube-controller-manager启动文件模板

#!/bin/bash
cd /data/k8s/work
source /data/k8s/bin/env.sh
cat > kube-controller-manager.service.template <<EOF
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
WorkingDirectory=${K8S_DIR}/kube-controller-manager
ExecStart=/data/k8s/bin/kube-controller-manager \\
  --profiling \\
  --cluster-name=kubernetes \\
  --controllers=*,bootstrapsigner,tokencleaner \\
  --kube-api-qps=1000 \\
  --kube-api-burst=2000 \\
  --leader-elect \\
  --use-service-account-credentials\\
  --concurrent-service-syncs=2 \\
  --bind-address=0.0.0.0 \\
  --secure-port=10257 \\
  --tls-cert-file=/etc/kubernetes/cert/kube-controller-manager.pem \\
  --tls-private-key-file=/etc/kubernetes/cert/kube-controller-manager-key.pem \\
  --port=10252 \\
  --authentication-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\
  --client-ca-file=/etc/kubernetes/cert/ca.pem \\
  --requestheader-allowed-names="" \\
  --requestheader-client-ca-file=/etc/kubernetes/cert/ca.pem \\
  --requestheader-extra-headers-prefix="X-Remote-Extra-" \\
  --requestheader-group-headers=X-Remote-Group \\
  --requestheader-username-headers=X-Remote-User \\
  --authorization-kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\
  --cluster-signing-cert-file=/etc/kubernetes/cert/ca.pem \\
  --cluster-signing-key-file=/etc/kubernetes/cert/ca-key.pem \\
  --experimental-cluster-signing-duration=876000h \\
  --horizontal-pod-autoscaler-sync-period=10s \\
  --concurrent-deployment-syncs=10 \\
  --concurrent-gc-syncs=30 \\
  --node-cidr-mask-size=24 \\
  --service-cluster-ip-range=${SERVICE_CIDR} \\
  --pod-eviction-timeout=6m \\
  --terminated-pod-gc-threshold=10000 \\
  --root-ca-file=/etc/kubernetes/cert/ca.pem \\
  --service-account-private-key-file=/etc/kubernetes/cert/ca-key.pem \\
  --kubeconfig=/etc/kubernetes/kube-controller-manager.kubeconfig \\
  --logtostderr=true \\
  --v=2
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF

配置详解

启动文件模板替换节点信息并分发到对应服务器

#!/bin/bsash
cd /data/k8s/work
source /data/k8s/bin/env.sh

# 替换
for (( i=0; i < 3; i++ ))
do
    sed -e "s/##NODE_NAME##/${MASTER_NAMES[i]}/" -e "s/##NODE_IP##/${MASTER_IPS[i]}/" kube-controller-manager.service.template > kube-controller-manager-${MASTER_IPS[i]}.service
done

# 分发
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    scp kube-controller-manager-${node_ip}.service root@${node_ip}:/etc/systemd/system/kube-controller-manager.service
done

启动控制器管理器

#!/bin/bash
source /data/k8s/bin/env.sh
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "mkdir -p ${K8S_DIR}/kube-controller-manager"
    ssh root@${node_ip} "systemctl daemon-reload && systemctl enable kube-controller-manager && systemctl restart kube-controller-manager"
done

验证

#!/bin/bash
source /data/k8s/bin/env.sh

# 进程check
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "systemctl status kube-controller-manager|grep -i Active"
done

# 端口check
for node_ip in ${MASTER_IPS[@]}
do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "netstat -antp|grep -E '10257|10252'"
done

验证结果
>>> 192.168.16.104
   Active: active (running) since Sun 2019-12-29 18:03:38 CST; 3 weeks 0 days ago
>>> 192.168.16.105
   Active: active (running) since Sat 2019-12-28 21:35:08 CST; 3 weeks 1 days ago
>>> 192.168.16.106
   Active: active (running) since Sat 2019-12-28 21:35:58 CST; 3 weeks 1 days ago
>>> 192.168.16.104
tcp 0      0 0.0.0.0:10252           0.0.0.0:*               LISTEN 754/kube-controller
tcp 0      0 0.0.0.0:10257           0.0.0.0:*               LISTEN 754/kube-controller
>>> 192.168.16.105
tcp 0      0 0.0.0.0:10252           0.0.0.0:*               LISTEN 11049/kube-controll
tcp 0      0 0.0.0.0:10257           0.0.0.0:*               LISTEN 11049/kube-controll
>>> 192.168.16.106
tcp 0      0 0.0.0.0:10252           0.0.0.0:*               LISTEN 11090/kube-controll
tcp 0      0 0.0.0.0:10257           0.0.0.0:*               LISTEN 11090/kube-controll
[root@master01 ~]#

总结

1. 控制器管理器管理控制器，各个控制器负责监视（watch）apiserver暴露的集群状态，并不断地尝试把当前状态向所期待的状态迁移；
2. 配置使用kubeconfig访问kube-apiserver安全端口；
3. 默认非安全端口10252，安全端口10257；
4. kube-controller-manager 3节点高可用，去竞争锁，成为leader；