一 介绍

     Etcd是一个高可用的 Key/Value 存储系统,主要用于分享配置和服务发现。

     简单:支持 curl 方式的用户 API (HTTP+JSON)
     安全:可选 SSL 客户端证书认证
     快速:单实例可达每秒 1000 次写操作
     可靠:使用 Raft 实现分布式

二 搭建开始

2.1 yum 安装etcd服务 (三台执行)

yum  -y install etcd

[root@k8s-master ~]# etcd -version
etcd Version: 3.3.11
Git SHA: 2cf9e51
Go Version: go1.10.3
Go OS/Arch: linux/amd64

2.2 安装cfssl工具,并配置证书:这里采用的是共用证书的方式(master执行)

mkdir  /etc/etcd/ssl

cd /etc/etcd/ssl

cat  etcd-root-ca-csr.json       #etcd根CA证书

{
"key": {
"algo": "rsa",
"size": 4096
},
"names": [
{
"O": "etcd",
"OU": "etcd Security",
"L": "Beijing",
"ST": "Beijing",
"C": "CN"
}
],
"CN": "etcd-root-ca"
}

cat etcd-gencert.json      #etcd集群证书

{
"signing": {
"default": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
}
}
}

cat  etcd-csr.json    #etcd集群证书

{
"key": {
"algo": "rsa",
"size": 4096
},
"names": [
{
"O": "etcd",
"OU": "etcd Security",
"L": "Beijing",
"ST": "Beijing",
"C": "CN"
}
],
"CN": "etcd",
"hosts": [
"127.0.0.1",
"localhost",
"192.168.137.66",
"192.168.137.16",
"192.168.137.26",
"k8s-master",
"k8s-node1",
"k8s-node2" -------->注意最后没有 ,
]
}

下载 cfssl

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
chmod +x cfssl_linux-amd64 cfssljson_linux-amd64
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson

生成证书

cfssl gencert --initca=true etcd-root-ca-csr.json | cfssljson --bare etcd-root-ca
cfssl gencert --ca etcd-root-ca.pem --ca-key etcd-root-ca-key.pem --config etcd-gencert.json etcd-csr.json | cfssljson --bare etcd

生成的文件列表如下 tree  .

.
├── etcd.csr
├── etcd-csr.json
├── etcd-gencert.json
├── etcd-key.pem
├── etcd.pem
├── etcd-root-ca.csr
├── etcd-root-ca-csr.json
├── etcd-root-ca-key.pem
├── etcd-root-ca.pem

2.3 分发证书(master执行)

I="192.168.137.16 192.168.137.26"

for IP in $I; do
ssh root@$IP mkdir /etc/etcd/ssl/
scp *.pem root@$IP:/etc/etcd/ssl/
ssh root@$IP chown -R etcd:etcd /etc/etcd/ssl/
ssh root@$IP chmod -R 755 /etc/etcd/
done #本台服务器也要设置权限
cd /etc/etcd/ssl
chown -R etcd:etcd /etc/etcd/ssl
chmod -R 755 /etc/etcd/ssl

2.4 修改配置文件(master执行)

[root@k8s-master ssl]# cat /etc/etcd/etcd.conf |grep -v '^#'
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.137.66:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.137.66:2379,http://127.0.0.1:2379"
ETCD_NAME="etcd01"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.137.66:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.137.66:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.137.66:2380,etcd02=https://192.168.137.16:2380,etcd03=https://192.168.137.26:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new" #注意状态为new
ETCD_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
ETCD_CLIENT_CERT_AUTH="True"
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-root-ca.pem"
ETCD_AUTO_TLS="True"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/etcd-key.pem"
ETCD_PEER_CLIENT_CERT_AUTH="True"
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/etcd-root-ca.pem"
ETCD_PEER_AUTO_TLS="True"

其他节点:注意上面的蓝色部分是要 修改的(其他节点上执行)

# k8s-node1
ETCD_LISTEN_PEER_URLS="https://192.168.137.16:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.137.16:2379,http://127.0.0.1:2379"
ETCD_NAME="etcd02"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.137.16:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.137.16:2379"
ETCD_INITIAL_CLUSTER_STATE="existing"  #注意这里,不能为new,有的是exist,但是这个版本测试为existing

# k8s-node2
ETCD_LISTEN_PEER_URLS="https://192.168.137.26:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.137.26:2379,http://127.0.0.1:2379"
ETCD_NAME="etcd02"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.137.26:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.137.26:2379"
ETCD_INITIAL_CLUSTER_STATE="existing"

2.5 启动(master先执行,其他两台后执行):master启动时间漫长,说明配置有问题

systemctl  daemon-reload

systemctl  start etcd

systemctl  enable  etcd

2.6 设置 etcdctl 的版本,有 v2和v3版本,他们的命令不同,这里采用v3版本

#export ETCDCTL_API=3
#cat /etc/profile
.....
export ETCDCTL_API=3

2.7 验证节点状态

etcdctl --cacert=/etc/etcd/ssl/etcd-root-ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem   \
--endpoints=https://192.168.137.66:2379,https://192.168.137.16:2379,https://192.168.137.26:2379 endpoint health
[root@k8s-master ssl]# etcdctl --cacert=/etc/etcd/ssl/etcd-root-ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem --endpoints=https://192.168.137.66:2379,https://192.168.137.16:2379,https://192.168.137.26:2379 endpoint health
https://192.168.137.16:2379 is healthy: successfully committed proposal: took = 4.807966ms
https://192.168.137.66:2379 is healthy: successfully committed proposal: took = 3.790949ms
https://192.168.137.66:2379 is healthy: successfully committed proposal: took = 2.493048ms

2.8 版本为2时对etcd的检查

export ETCDCTL_API=2

[root@k8s-master ssl]# etcdctl --ca-file=/etc/etcd/ssl/etcd-root-ca.pem --cert-file=/etc/etcd/ssl/etcd.pem --key-file=/etc/etcd/ssl/etcd-key.pem  \
--endpoints=https://192.168.137.66:2379,https://192.168.137.16:2379 cluster-health
member 457528f516aae01a is healthy: got healthy result from https://192.168.137.66:2379
member b13478c4279881c2 is healthy: got healthy result from https://192.168.137.16:2379
cluster is healthy

三 报错以及解决

error 1:执行etcdctl命令检查时报错

etcdctl --cacert=/etc/etcd/ssl/etcd-root-ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem   \
--endpoints=https://192.168.137.66:2379,https://192.168.137.16:2379,https://192.168.137.26:2379 endpoint health
........
flag provided but not defined: -cacert

solution:版本不同,命令的格式不同

export ETCDCTL_API=3

error 2: failed to check the health of member 6c70a880257288f on https://192.168.137.16:2379: Get https://192.168.137.16:2379/health: remote error: tls: bad certificate

solution:证书问题

重做步骤2.2和2.3

error 3: couldn't find local name "etcd04" in the initial cluster configuration

solution:配置文件问题

检查步骤2.4,着重看以下方面
ETCD_NAME="etcd01"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.137.66:2380,etcd02=https://192.168.137.16:2380,etcd03=https://192.168.137.26:2380" 接着执行下面的命令
systemctl stop etcd
rm -rf /var/lib/etcd/default.etcd
systemctl daemon-reload && systemctl restart etcd

error 4:错误如下

[root@k8s-node1 ~]# etcdctl member list
Error: dial tcp 127.0.0.1:2379: connect: connection refused

solution:根据步骤2.4修改

ETCD_LISTEN_CLIENT_URLS="https://192.168.137.16:2379,http://127.0.0.1:2379"

[root@k8s-node1 ~]# etcdctl member list
457528f516aae01a, started, etcd01, https://192.168.137.66:2380, https://192.168.137.66:2379
b13478c4279881c2, started, etcd02, https://192.168.137.16:2380, https://192.168.137.16:2379

a93278c4200188c5, started, etcd03, https://192.168.137.26:2380, https://192.168.137.26:2379

k8s集群搭建之二:etcd集群的搭建的更多相关文章

  1. linux下oracle11G DG搭建(二):环绕主库搭建操作

    linux下oracle11G DG搭建(二):环绕主库搭建操作 环境 名称 主库 备库 主机名 bjsrv shsrv 软件版本号 RedHat Enterprise5.5.Oracle 11g 1 ...

  2. kubeadm 线上集群部署(一) 外部 ETCD 集群搭建

      IP Hostname   192.168.1.23 k8s-etcd-01 etcd集群节点,默认关于ETCD所有操作均在此节点上操作 192.168.1.24 k8s-etcd-02 etcd ...

  3. 使用k8s operator安装和维护etcd集群

    关于Kubernetes Operator这个新生事物,可以参考下文来了解这一技术的来龙去脉: https://yq.aliyun.com/articles/685522?utm_content=g_ ...

  4. 彻底搞懂 etcd 系列文章(三):etcd 集群运维部署

    0 专辑概述 etcd 是云原生架构中重要的基础组件,由 CNCF 孵化托管.etcd 在微服务和 Kubernates 集群中不仅可以作为服务注册与发现,还可以作为 key-value 存储的中间件 ...

  5. 使用Kubeadm搭建高可用Kubernetes集群

    1.概述 Kubenetes集群的控制平面节点(即Master节点)由数据库服务(Etcd)+其他组件服务(Apiserver.Controller-manager.Scheduler...)组成. ...

  6. kubernetes部署 etcd 集群

    本文档介绍部署一个三节点高可用 etcd 集群的步骤: etcd 集群各节点的名称和 IP 如下: kube-node0:192.168.111.10kube-node1:192.168.111.11 ...

  7. 使用docker配置etcd集群

    docker配置etcd集群与直接部署etcd集群在配置上并没有什么太大差别. 我这里直接使用docker-compose来实现容器化的etcd部署 环境如下: HostName IP etcd1 1 ...

  8. Kubernetes(k8s)集群部署(k8s企业级Docker容器集群管理)系列之自签TLS证书及Etcd集群部署(二)

    0.前言 整体架构目录:ASP.NET Core分布式项目实战-目录 k8s架构目录:Kubernetes(k8s)集群部署(k8s企业级Docker容器集群管理)系列目录 一.服务器设置 1.把每一 ...

  9. Etcd学习(二)集群搭建Clustering

    1.单个etcd节点(测试开发用) 之前我一直开发测试一直是用的一个Etcd节点,然后启动命令一直都是直接打一个etcd(我已经将etcd安装目录的bin目录加入到PATH环 境变量中),然后启动信息 ...

随机推荐

  1. ios微信分享的兼容性问题

    我微信分享采用的是: 页面初始化时动态加载js-sdk, 然后在需要分享的页面进行sdk的分享初始化 app.vue store.vue 这种方法在安卓上完全正常, 好用得令人发指, 但是!!! io ...

  2. 小程序内嵌H5页面判断微信及小程序环境

    判断微信及小程序环境 1.H5页面引入jweixin-1.3.2.js 2. var ua = window.navigator.userAgent.toLowerCase(); if(ua.matc ...

  3. Python文件读写(一)

    import time as t from os import path def createFile(dest): date = t.localtime(t.time()) name = '%d_% ...

  4. UNP学习第九章 基本名字与地址转换

    之前都用数值地址来表示主机(206.6.226.33),用数值端口号来标识服务器. 然而,我们应该使用名字而不是数值:名字比较容易记,数值地址可以改变但名字保持不变. 随着往IPv6上转移,数值地址变 ...

  5. QT blockingmaster例子学习

    dialog.h: #ifndef DIALOG_H #define DIALOG_H #include <QDialog> #include "masterthread.h&q ...

  6. 工程师技术(四):配置SMB文件夹共享、多用户Samba挂载、普通NFS共享的实现、安全NFS共享的实现

    一.配置SMB文件夹共享 目标: 本例要求在虚拟机 server0 上发布两个共享文件夹,具体要求如下: 1> 此服务器必须是 STAFF 工作组的一个成员   2> 发布目录 /comm ...

  7. cms网站模板收集

    Dedecms模板 1.魔客吧  -----  dedecms .wordpress .discuz.shopnc.ecshop等模板都有 eg: html5绿色响应式企业网页模板 2.绘艺素材 -- ...

  8. getstu

    #coding: utf- #title..href... import urllib.request import time url=[ page= j= : url[j]='http://www. ...

  9. JS-text节点模拟innerHTML属性

    # [在线预览](https://jsfiddle.net/1010543618/mz7ybu8g/2/) text 节点无 innerHTML 这个属性!!! 如果直接修改 text 节点的属性(d ...

  10. 88、使用tensorboard进行可视化学习,查看具体使用时间,训练轮数,使用内存大小

    ''' Created on 2017年5月23日 @author: weizhen ''' import os import tensorflow as tf from tensorflow.exa ...