Kubernetes集群搭建之Etcd集群配置篇

介绍

---

etcd 是一个分布式一致性k-v存储系统，可用于服务注册发现与共享配置，具有以下优点。

简单 ： 相比于晦涩难懂的paxos算法，etcd基于相对简单且易实现的raft算法实现一致性，并通过gRPC提供接口调用

安全：支持TLS通信，并可以针对不同的用户进行对key的读写控制

高性能：10,000 /秒的写性能

本次系列使用的所需部署包版本都使用的目前最新的或最新稳定版，安装包地址请到公众号内回复【K8s实战】获取

注: K8s的集群相关数据是存放在Etcd中的,所有我们得先部署

架构

---

cfssl安装

---

Kubernetes 系统的各组件需要使用 TLS 证书对通信进行加密，使用 CloudFlare 的 PKI 工具集 cfssl 来生成 Certificate Authority (CA) 和其它证书。

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64mv cfssl_linux-amd64 /usr/local/bin/cfsslmv cfssljson_linux-amd64 /usr/local/bin/cfssljsonmv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

创建证书目录

---

[root@master-01 ssl]#mkdir /etc/etcd/ssl -p[root@master-01 ssl]#mkdir /etc/kubernetes/ssl -p[root@master-01 ssl]##cd /etc/etcd/ssl

生成证书

---

1. Etcd ca配置

[root@master-01 ssl]# cat << EOF | tee ca-config.json{"signing": {"default": {"expiry": "87600h"},"profiles": {"etcd": {"expiry": "87600h","usages": ["signing","key encipherment","server auth","client auth"]}}}}EOF

2. Etcd ca证书

[root@master-01 ssl]# cat << EOF | tee ca-csr.json{"CN": "etcd CA","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "Hangzhou","ST": "Hangzhou"}]}EOF

3. etcd server证书

[root@master-01 ssl]# cat << EOF | tee server-csr.json{"CN": "etcd","hosts": ["192.168.209.130","192.168.209.131","192.168.209.132"],"key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "Hangzhou","ST": "Hangzhou"}]}EOF

3. 生成etcd ca证书和私钥 初始化ca

[root@master-01 ssl]# lsca-config.json ca-csr.json server-csr.json[root@master-01 ssl]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca2019/03/09 14:49:51 [INFO] generating a new CA key and certificate from CSR2019/03/09 14:49:51 [INFO] generate received request2019/03/09 14:49:51 [INFO] received CSR2019/03/09 14:49:51 [INFO] generating key: rsa-20482019/03/09 14:49:51 [INFO] encoded CSR2019/03/09 14:49:51 [INFO] signed certificate with serial number 131013203369168241950883398321469825148900357407[root@master-01 ssl]# lsca-config.json ca.csr ca-csr.json ca-key.pem ca.pem server-csr.json

****4.  生成server 证书

[root@master-01 ssl]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd server-csr.json | cfssljson -bare server2019/03/09 14:53:18 [INFO] generate received request2019/03/09 14:53:18 [INFO] received CSR2019/03/09 14:53:18 [INFO] generating key: rsa-20482019/03/09 14:53:18 [INFO] encoded CSR2019/03/09 14:53:18 [INFO] signed certificate with serial number 1346689003013978487615203135781755822261954464372019/03/09 14:53:18 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable forwebsites. For more information see the Baseline Requirements for the Issuance and Managementof Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);specifically, section 10.2.3 ("Information Requirements").[root@master-01 ssl]# lsca-config.json ca.csr ca-csr.json ca-key.pem ca.pem server.csr server-csr.json server-key.pem server.pem

以上生成etcd证书完成，只需要会使用即可，不用过多深入研究

同步证书

---

拷贝证书 master-02、master-03上

[root@master-01 ssl]# scp * 192.168.209.131:/etc/etcd/ssl/[root@master-01 ssl]# scp * 192.168.209.132:/etc/etcd/ssl/

etcd 部署

---

1. 解压缩并同步二进制文件到 master-02、master-03

[root@master-01 ~]# tar xf etcd-v3.3.12-linux-amd64.tar.gz [root@master-01 ~]# cd etcd-v3.3.12-linux-amd64[root@master-01 etcd-v3.3.12-linux-amd64]# cp etcd etcdctl /usr/bin/[root@master-01 etcd-v3.3.12-linux-amd64]# scp etcd etcdctl 192.168.209.131:/usr/bin/[root@master-01 etcd-v3.3.12-linux-amd64]# scp etcd etcdctl 192.168.209.132:/usr/bin/

2. 创建etcd数据目录

[root@master-01 ~]# mkdir /var/lib/etcd

3. 配置etcd文件

[root@master-01 ~]# cat /etc/etcd/etcd.conf # configure file for etcd.service#[Member]ETCD_NAME="infra1" ETCD_DATA_DIR="/var/lib/etcd/infra1.etcd"ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380"ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379"#[Clustering]ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.209.130:2380"ETCD_ADVERTISE_CLIENT_URLS="https://192.168.209.130:2379"ETCD_INITIAL_CLUSTER="infra1=https://192.168.209.130:2380,infra2=https://192.168.209.131:2380,infra3=https://192.168.209.132:2380"ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"ETCD_INITIAL_CLUSTER_STATE="new"#[Security]ETCD_CERT_FILE="/etc/etcd/ssl/server.pem"ETCD_KEY_FILE="/etc/etcd/ssl/server-key.pem"ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.pem"ETCD_CLIENT_CERT_AUTH="true"ETCD_PEER_CERT_FILE="/etc/etcd/ssl/server.pem"ETCD_PEER_KEY_FILE="/etc/etcd/ssl/server-key.pem"ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.pem"ETCD_PEER_CLIENT_CERT_AUTH="true"ETCD_PEER_CERT_FILE="/etc/etcd/ssl/server.pem"ETCD_PEER_KEY_FILE="/etc/etcd/ssl/server-key.pem"ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.pem"ETCD_PEER_CLIENT_CERT_AUTH="true"

ETCD_NAME 节点名称

ETCD_DATA_DIR 数据目录

ETCD_LISTEN_PEER_URLS 集群通信监听地址

ETCD_LISTEN_CLIENT_URLS 客户端访问监听地址

ETCD_INITIAL_ADVERTISE_PEER_URLS 集群通告地址

ETCD_ADVERTISE_CLIENT_URLS 客户端通告地址

ETCD_INITIAL_CLUSTER 集群节点地址

ETCD_INITIAL_CLUSTER_TOKEN 集群Token

ETCD_INITIAL_CLUSTER_STATE 加入集群的当前状态，new是新集群，existing表示加入已有集群

注意:  master-02、master-03节点上需要修改ETCD_NAME 和对应IP地址

4.. 配置etcd启动文件

[root@master-01 ~]# cat /usr/lib/systemd/system/etcd.service

[Unit]Description=Etcd ServerAfter=network.targetAfter=network-online.targetWants=network-online.target[Service]Type=notifyWorkingDirectory=/var/lib/etcd/EnvironmentFile=-/etc/etcd/etcd.conf# set GOMAXPROCS to number of processorsExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/etcd --name=\"${ETCD_NAME}\" --data-dir=\"${ETCD_DATA_DIR}\" --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\" --listen-peer-urls=\"${ETCD_LISTEN_PEER_URLS}\" --advertise-client-urls=\"${ETCD_ADVERTISE_CLIENT_URLS}\" --initial-cluster-token=\"${ETCD_INITIAL_CLUSTER_TOKEN}\" --initial-cluster=\"${ETCD_INITIAL_CLUSTER}\" --initial-cluster-state=\"${ETCD_INITIAL_CLUSTER_STATE}\" --cert-file=\"${ETCD_CERT_FILE}\" --key-file=\"${ETCD_KEY_FILE}\" --trusted-ca-file=\"${ETCD_TRUSTED_CA_FILE}\" --client-cert-auth=\"${ETCD_CLIENT_CERT_AUTH}\" --peer-cert-file=\"${ETCD_PEER_CERT_FILE}\" --peer-key-file=\"${ETCD_PEER_KEY_FILE}\" --peer-trusted-ca-file=\"${ETCD_PEER_TRUSTED_CA_FILE}\" --peer-client-cert-auth=\"${ETCD_PEER_CLIENT_CERT_AUTH}\""Restart=on-failureLimitNOFILE=65536[Install]WantedBy=multi-user.target

5. 同步配置及启动文件到 master-02、master-03

[root@master-01 ~]# scp /etc/etcd/etcd.conf 192.168.209.132:/etc/etcd/[root@master-01 ~]# scp /etc/etcd/etcd.conf 192.168.209.131:/etc/etcd/[root@master-01 ~]# scp /usr/lib/systemd/system/etcd.service 192.168.209.131:/usr/lib/systemd/system/[root@master-01 ~]# scp /usr/lib/systemd/system/etcd.service 192.168.209.132:/usr/lib/systemd/system/

6. 启动etcd，注意02 03 要同样配置下

[root@master-01 ssl]# systemctl daemon-reload[root@master-01 ssl]# systemctl enable etcd[root@master-01 ssl]# systemctl start etcd

7. 验证集群状态

[root@master-01 ~]# etcdctl --ca-file=/etc/etcd/ssl/ca.pem --cert-file=/etc/etcd/ssl/server.pem --key-file=/etc/etcd/ssl/server-key.pem --endpoints="https://192.168.209.130:2379,https://192.168.209.131:2379,https://192.168.209.132:2379" cluster-healthmember 586a62bfa8ca5f8a is healthy: got healthy result from https://192.168.209.130:2379member 5f55d6687099cd91 is healthy: got healthy result from https://192.168.209.131:2379member ee591e661050ac9d is healthy: got healthy result from https://192.168.209.132:2379cluster is healthy

进行到这一步，说明etcd集群部署完成啦，下一章介绍部署高可用K8s集群，敬请期待，谢谢！

END

如果你觉得文章还不错，请大家点『好看』分享下。你的肯定是我最大的鼓励和支持。