k8s-高可用多主master配置
- 准备主机
- centos7镜像
- node1: 192.168.0.101
- node2: 192.168.0.102
- node3: 192.168.0.103
- vip: 192.168.0.104
- 配置ssh免密 并修改/etc/hosts跟/etc/hostname
- 配置所有节点的kubelet
# 配置kubelet使用国内可用镜像
# 修改/etc/systemd/system/kubelet.service.d/-kubeadm.conf
# 添加如下配置
Environment="KUBELET_EXTRA_ARGS=--pod-infra-container-image=registry.cn-shanghai.aliyuncs.com/gcr-k8s/pause-amd64:3.0"
# 使用命令
sed -i '/ExecStart=$/i Environment="KUBELET_EXTRA_ARGS=--pod-infra-container-image=registry.cn-shanghai.aliyuncs.com/gcr-k8s/pause-amd64:3.0"' /etc/systemd/system/kubelet.service.d/-kubeadm.conf
# 重新载入配置
systemctl daemon-reload
- 修改环境变量
rm -rf $HOME/.kube
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config systemctl stop firewalld && systemctl disable firewalld setenforce #临时禁用selinux
sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/sysconfig/selinux #永久关闭 修改/etc/sysconfig/selinux文件设置
sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config swapoff -a #临时关闭swap
sed -i 's/.*swap.*/#&/' /etc/fstab #永久关闭 注释/etc/fstab文件里swap相关的行 cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables =
net.bridge.bridge-nf-call-iptables =
EOF
sysctl --system cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=
gpgcheck=
repo_gpgcheck=
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF yum install -y kubelet-1.11. kubeadm-1.11. kubectl-1.11.
- 拉取镜像
＃新建脚本
images=(kube-proxy-amd64:v1.11.0 kube-scheduler-amd64:v1.11.0 kube-controller-manager-amd64:v1.11.0 kube-apiserver-amd64:v1.11.0
etcd-amd64:3.2. coredns:1.1. pause-amd64:3.1 kubernetes-dashboard-amd64:v1.8.3 k8s-dns-sidecar-amd64:1.14. k8s-dns-kube-dns-amd64:1.14.
k8s-dns-dnsmasq-nanny-amd64:1.14. )
for imageName in ${images[@]} ; do
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
done
- 部署keepalived跟haproxy
# 新建脚本 每台master执行
docker pull haproxy:1.7.-alpine
mkdir /etc/haproxy
cat >/etc/haproxy/haproxy.cfg<<EOF
global
log 127.0.0.1 local0 err
maxconn
uid
gid
#daemon
nbproc
pidfile haproxy.pid defaults
mode http
log 127.0.0.1 local0 err
maxconn
retries
timeout connect 5s
timeout client 30s
timeout server 30s
timeout check 2s listen admin_stats
mode http
bind 0.0.0.0:
log 127.0.0.1 local0 err
stats refresh 30s
stats uri /haproxy-status
stats realm Haproxy\ Statistics
stats auth will:will
stats hide-version
stats admin if TRUE frontend k8s-https
bind 0.0.0.0:
mode tcp
#maxconn
default_backend k8s-https backend k8s-https
mode tcp
balance roundrobin
server k8s-master01 172.16.2.71: weight maxconn check inter rise fall
server k8s-master02 172.16.2.72: weight maxconn check inter rise fall
server k8s-master03 172.16.2.73: weight maxconn check inter rise fall
EOF # 启动haproxy
docker run -d --name my-haproxy \
-v /etc/haproxy:/usr/local/etc/haproxy:ro \
-p : \
-p : \
--restart always \
haproxy:1.7.-alpine #keepalived
docker pull osixia/keepalived:1.4.
docker run --net=host --cap-add=NET_ADMIN \
-e KEEPALIVED_INTERFACE=ens33 \ #改成你的网卡名称
-e KEEPALIVED_VIRTUAL_IPS="#PYTHON2BASH:['192.168.0.104']" \
-e KEEPALIVED_UNICAST_PEERS="#PYTHON2BASH:['192.168.0.101,'192.168.0.102','192.168.0.103']" \
-e KEEPALIVED_PASSWORD=admin \
--name k8s-keepalived \
--restart always \
-d osixia/keepalived:1.4.
- 启动第一台master
# 新建脚本 执行完成后保证master启动日志 含有node节点加入token命令
LOAD_BALANCER_DNS="192.168.0.101"
LOAD_BALANCER_PORT=""
CP0_HOSTNAME="node1"
CP0_IP="192.168.0.104"
cat >kubeadm-config.yaml<<EOF
apiVersion: kubeadm.k8s.io/v1alpha2
kind: MasterConfiguration
kubernetesVersion: v1.11.0
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers apiServerCertSANs:
- "$LOAD_BALANCER_DNS"
- "node1"
- "node2"
- "node3"
- "192.168.0.101"
- "192.168.0.102"
- "192.168.0.103"
- "192.168.0.104"
- "127.0.0.1"
api:
controlPlaneEndpoint: "$LOAD_BALANCER_DNS:$LOAD_BALANCER_PORT"
etcd:
local:
extraArgs:
listen-client-urls: "https://127.0.0.1:2379,https://$CP0_IP:2379"
advertise-client-urls: "https://$CP0_IP:2379"
listen-peer-urls: "https://$CP0_IP:2380"
initial-advertise-peer-urls: "https://$CP0_IP:2380"
initial-cluster: "$CP0_HOSTNAME=https://$CP0_IP:2380"
serverCertSANs:
- $CP0_HOSTNAME
- $CP0_IP
peerCertSANs:
- $CP0_HOSTNAME
- $CP0_IP
networking:
# This CIDR is a Calico default. Substitute or remove for your CNI provider.
podSubnet: "192.168.0.0/16"
EOF kubeadm init --config kubeadm-config.yaml > start.log cd /etc/kubernetes && tar cvzf k8s-key.tgz admin.conf pki/ca.* pki/sa.* pki/front-proxy-ca.* pki/etcd/ca.* scp k8s-key.tgz node2:~/ scp k8s-key.tgz node1:~/ ssh node2 'tar xf k8s-key.tgz -C /etc/kubernetes/' ssh node1 'tar xf k8s-key.tgz -C /etc/kubernetes/' rm -rf $HOME/.kube
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
- 启动第二台master
#新建脚本
LOAD_BALANCER_DNS="192.168.0.104"
LOAD_BALANCER_PORT=""
CP0_HOSTNAME="node1"
CP0_IP="192.168.0.101"
CP1_HOSTNAME="node2"
CP1_IP="192.168.0.102"
cat >kubeadm-config.yaml<<EOF
apiVersion: kubeadm.k8s.io/v1alpha2
kind: MasterConfiguration
kubernetesVersion: v1.11.0
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
apiServerCertSANs:
- "$LOAD_BALANCER_DNS"
- "node1"
- "node2"
- "node3"
- "192.168.0.101"
- "192.168.0.102"
- "192.168.0.103"
- "192.168.0.104"
- "127.0.0.1"
api:
controlPlaneEndpoint: "$LOAD_BALANCER_DNS:$LOAD_BALANCER_PORT"
etcd:
local:
extraArgs:
listen-client-urls: "https://127.0.0.1:2379,https://$CP1_IP:2379"
advertise-client-urls: "https://$CP1_IP:2379"
listen-peer-urls: "https://$CP1_IP:2380"
initial-advertise-peer-urls: "https://$CP1_IP:2380"
initial-cluster: "$CP0_HOSTNAME=https://$CP0_IP:2380,$CP1_HOSTNAME=https://$CP1_IP:2380"
initial-cluster-state: existing
serverCertSANs:
- $CP1_HOSTNAME
- $CP1_IP
peerCertSANs:
- $CP1_HOSTNAME
- $CP1_IP
networking:
# This CIDR is a calico default. Substitute or remove for your CNI provider.
podSubnet: "192.168.0.0/16"
EOF kubeadm alpha phase certs all --config kubeadm-config.yaml
kubeadm alpha phase kubelet config write-to-disk --config kubeadm-config.yaml
kubeadm alpha phase kubelet write-env-file --config kubeadm-config.yaml
kubeadm alpha phase kubeconfig kubelet --config kubeadm-config.yaml
systemctl start kubelet
KUBECONFIG=/etc/kubernetes/admin.conf kubectl exec -n kube-system etcd-${CP0_HOSTNAME} -- etcdctl --ca-file /etc/kubernetes/pki/etcd/ca.crt --cert-file /etc/kubernetes/pki/etcd/peer.crt --key-file /etc/kubernetes/pki/etcd/peer.key --endpoints=https://${CP0_IP}:2379 member add ${CP1_HOSTNAME} https://${CP1_IP}:2380
kubeadm alpha phase etcd local --config kubeadm-config.yaml
kubeadm alpha phase kubeconfig all --config kubeadm-config.yaml
kubeadm alpha phase controlplane all --config kubeadm-config.yaml
kubeadm alpha phase mark-master --config kubeadm-config.yaml
- 部署第三台master
# 新建脚本
LOAD_BALANCER_DNS="192.168.91.100"
LOAD_BALANCER_PORT=""
CP0_HOSTNAME="node1"
CP0_IP="192.168.0.101"
CP1_HOSTNAME="node2"
CP1_IP="192.168.0.102"
CP2_HOSTNAME="node3"
CP2_IP="192.168.0.103"
cat >kubeadm-config.yaml<<EOF
apiVersion: kubeadm.k8s.io/v1alpha2
kind: MasterConfiguration
kubernetesVersion: v1.11.0
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
piServerCertSANs:
- "$LOAD_BALANCER_DNS"
- "node1"
- "node2"
- "node3"
- "192.168.0.101"
- "192.168.0.102"
- "192.168.0.103"
- "192.168.0.104"
- "127.0.0.1"
api:
controlPlaneEndpoint: "$LOAD_BALANCER_DNS:$LOAD_BALANCER_PORT"
etcd:
local:
extraArgs:
listen-client-urls: "https://127.0.0.1:2379,https://$CP2_IP:2379"
advertise-client-urls: "https://$CP2_IP:2379"
listen-peer-urls: "https://$CP2_IP:2380"
initial-advertise-peer-urls: "https://$CP2_IP:2380"
initial-cluster: "$CP0_HOSTNAME=https://$CP0_IP:2380,$CP1_HOSTNAME=https://$CP1_IP:2380,$CP2_HOSTNAME=https://$CP2_IP:2380"
initial-cluster-state: existing
serverCertSANs:
- $CP2_HOSTNAME
- $CP2_IP
peerCertSANs:
- $CP2_HOSTNAME
- $CP2_IP
networking:
# This CIDR is a calico default. Substitute or remove for your CNI provider.
podSubnet: "192.168.0.0/16"
EOF kubeadm alpha phase certs all --config kubeadm-config.yaml
kubeadm alpha phase kubelet config write-to-disk --config kubeadm-config.yaml
kubeadm alpha phase kubelet write-env-file --config kubeadm-config.yaml
kubeadm alpha phase kubeconfig kubelet --config kubeadm-config.yaml
systemctl start kubelet
KUBECONFIG=/etc/kubernetes/admin.conf kubectl exec -n kube-system etcd-${CP0_HOSTNAME} -- etcdctl --ca-file /etc/kubernetes/pki/etcd/ca.crt --cert-file /etc/kubernetes/pki/etcd/peer.crt --key-file /etc/kubernetes/pki/etcd/peer.key --endpoints=https://${CP0_IP}:2379 member add ${CP2_HOSTNAME} https://${CP2_IP}:2380
kubeadm alpha phase etcd local --config kubeadm-config.yaml
kubeadm alpha phase kubeconfig all --config kubeadm-config.yaml
kubeadm alpha phase controlplane all --config kubeadm-config.yaml
kubeadm alpha phase mark-master --config kubeadm-config.yaml
- 部署网络可以使用calico或者flannel
#新建脚本
wget https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/hosted/rbac-kdd.yaml wget https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml
＃每个master节点执行 calico使用DaemonSet部署到每个node
kubectl taint nodes --all node-role.kubernetes.io/master-
kubectl apply -f rbac-kdd.yaml
kubectl apply -f calico.yaml
＃查看master节点状况
kubectl get pods --all-namespaces
- 加入node节点
＃第一台master执行完 start．log找出token命令
kubeadm join 192.168.0.104: --token v8q1a2.9zb56bff4076tmin --discovery-token-ca-cert-hash sha256:94a1bc52ce95cb8a69c97528d81ca0ea2bde48947450b9c7b59225dcafe8cebc
k8s-高可用多主master配置的更多相关文章
- .Net Core2.1 秒杀项目一步步实现CI/CD(Centos7.2)系列一:k8s高可用集群搭建总结以及部署API到k8s
前言:本系列博客又更新了,是博主研究很长时间,亲自动手实践过后的心得,k8s集群是购买了5台阿里云服务器部署的,这个集群差不多搞了一周时间,关于k8s的知识点,我也是刚入门,这方面的知识建议参考博客园 ...
- Nginx+keepalived(高可用双主模式)
Nginx+keepalived(高可用双主模式) tips:前面已经介绍了nginx+keepalived高可用主从模式,今天补充下高可用的双主模式,均可以作为主机使用 server1:192.16 ...
- Redis高可用之集群配置（六）
0.Redis目录结构 1)Redis介绍及部署在CentOS7上(一) 2)Redis指令与数据结构(二) 3)Redis客户端连接以及持久化数据(三) 4)Redis高可用之主从复制实践(四) 5 ...
- hadoop HA+Federation（高可用联邦）搭建配置（二）
hadoop HA+Federation(高可用联邦)搭建配置(二) 标签(空格分隔): hadoop core-site.xml <?xml version="1.0" e ...
- hadoop HA+Federation（高可用联邦）搭建配置（一）
hadoop HA+Federation(高可用联邦)搭建配置(一) 标签(空格分隔): 未分类 介绍 hadoop 集群一共有4种部署模式,详见<hadoop 生态圈介绍>. HA联邦模 ...
- kubeadm实现k8s高可用集群环境部署与配置
高可用架构 k8s集群的高可用实际是k8s各核心组件的高可用,这里使用主备模式,架构如下: 主备模式高可用架构说明: 核心组件 高可用模式 高可用实现方式 apiserver 主备 keepalive ...
- python安装二进制k8s高可用 版本1.13.0
一.所有安装包.脚本.脚本说明.下载链接:https://pan.baidu.com/s/1kHaesJJuMQ5cG-O_nvljtg 提取码:kkv6 二.脚本安装说明 1.脚本说明: 本实验为三 ...
- 【葵花宝典】lvs+keepalived部署kubernetes(k8s)高可用集群
一.部署环境 1.1 主机列表 主机名 Centos版本 ip docker version flannel version Keepalived version 主机配置 备注 lvs-keepal ...
- Haproxy+Keepalived高可用负载均衡详细配置
本文所使用的环境: 10.6.2.128 centos6.5 10.6.2.129 centos6.5 VIP 为10.6.2.150 要实现的目标: 实现10.6.2.128和10.6 ...
随机推荐
- springboot无法获取证书内容
最近项目里面在接第三方验证的时候,需要用到生成的公钥和私钥证书.在demo测试的时候,发现在resources里面直接建立一个key文件夹放入证书文件,然后使用文件方式去获取,大概代码如下: File ...
- [Abp vNext微服务实践] - 添加中文语言
简介 abp vNext中提供了多语言功能,默认语言是英文,没有提供中文语言包.在业务开发中,定义权限后需要用中文的备注提供角色选择,本篇将介绍如何在abp vNext中加入中文语言. step1:添 ...
- 区块链火爆，再不知道Golang就晚了
Golang,也叫Go语言,是2009年刚刚被发发布的一门新语言. 区块链,是2019年我国提出的新战略. 一个不争的事实就是,大多数从事区块链开发的小伙伴都是用Golang,大多数招聘区块链技术工作 ...
- linux——在windows上搭建linux练习环境
程序员自己研究——java-linux-php——环境搭建 需要首选准备一个linux环境. 1,可用安装一个虚拟机:VMware虚拟机 2,安装一个VMware大约5分钟左右. 3,截止目前2019 ...
- 最近在研究IO
import java.io.File; import java.io.IOException; public class Demo11_1 { public static void main(Str ...
- 1.打开windows中功能的快捷方式
1.打开组策略 命令:gpedit.msc 2.打开注册表 命令:regedit 3.快速打开本地安全组策略 命令:secpol.msc 4.打开服务 命令:services.msc 5.系统退域的时 ...
- node.js http模块和fs模块上机实验·
httpserver const httpserver = require('http'); var server = httpserver.createServer(function (req,re ...
- java读取 xml文件
java读取xml文件的四种方法 转自https://www.cnblogs.com/lingyao/p/5708929.html Xml代码 1 <?xml version="1. ...
- (vue.js)Vue element tab 每个tab用一个路由来管理？
(vue.js)Vue element tab 每个tab用一个路由来管理? 来源:网络整理 时间:2017/5/13 0:24:01 关键词: 关于网友提出的“ (vue.js) ...
- sublime中替换成换行
ctrl + h 打开替换窗口. replace with里输入 ctrl + shift + enter