This document describes how to deploy kubernetes on multiple ubuntu nodes, including 1 master node and 3 minion nodes, and people uses this approach can scale to any number of minion nodes by changing some settings with ease. Although there exists saltstack based ubuntu k8s installation , it may be tedious and hard for a guy that knows little about saltstack but want to build a really distributed k8s cluster. This approach is inspired by k8s deploy on a single node.

Cloud team from ZJU will keep updating this work.

Prerequisites:

1 The minion nodes have installed docker version 1.2+

2 All machines can communicate with each orther, no need to connect Internet (should use private docker registry in this case)

3 These guide is tested OK on Ubuntu 14.04 LTS 64bit server, but it should also work on most Ubuntu versions

4 Dependences of this guide: etcd-2.0.0, flannel-0.2.0, k8s-0.12.0, but it may work with higher versions

Main Steps

I. Make kubernetes , etcd and flanneld binaries

On your laptop, copy cluster/ubuntu-cluster directory to your workspace.

The build.sh will download and build all the needed binaries into ./binaries.

You can customize your etcd version or K8s version in the build.sh by changing variable ETCD_V and K8S_V in build.sh, default etcd version is 2.0.0 and K8s version is 0.12.0.

$ cd cluster/ubuntu-cluster
$ sudo ./build.sh

Please copy all the files in ./binaries into /opt/bin of every machine you want to run as Kubernetes cluster node.

Alternatively, if your Kubernetes nodes have access to Internet, you can copy cluster/ubuntu-cluster directory to every node and run:

# in every node
$ cd cluster/ubuntu-cluster
$ sudo ./build.sh
$ sudo cp ./binaries/* /opt/bin

We used flannel here because we want to use overlay network, but please remember it is not the only choice, and it is also not a k8s' necessary dependence. Actually you can just build up k8s cluster natively, or use flannel, Open vSwitch or any other SDN tool you like, we just choose flannel here as a example.

II. Configue and install every components upstart script

An example cluster is listed as below:

IP Address Role
10.10.103.223 minion
10.10.103.224 minion
10.10.103.162 minion
10.10.103.250 master

First of all, make sure cluster/ubuntu-cluster exists on this node,and run configue.sh.

On master( infra1 10.10.103.250 ) node:

# in cluster/ubuntu-cluster
$ sudo ./configure.sh
Welcome to use this script to configure k8s setup Please enter all your cluster node ips, MASTER node comes first
And separated with blank space like "<ip_1> <ip2> <ip3>": 10.10.103.250 10.10.103.223 10.10.103.224 10.10.103.162 This machine acts as
both MASTER and MINION: 1
only MASTER: 2
only MINION: 3
Please choose a role > 2 IP address of this machine > 10.10.103.250 Configure Success

On every minion ( e.g. 10.10.103.224 ) node:

# in cluster/ubuntu-cluster
$ sudo ./configure.sh
Welcome to use this script to configure k8s setup Please enter all your cluster node ips, MASTER node comes first
And separated with blank space like "<ip_1> <ip2> <ip3>": 10.10.103.250 10.10.103.223 10.10.103.224 10.10.103.162 This machine acts as
both MASTER and MINION: 1
only MASTER: 2
only MINION: 3
Please choose a role > 3 IP address of this machine > 10.10.103.224 Configure Success

If you want a node acts as both running the master and minion, please choose option 1.

III. Start all components

  1. On the master node:

    $ sudo service etcd start

    Then on every minion node:

    $ sudo service etcd start

    The kubernetes commands will be started automatically after etcd

  2. On any node:

    $ /opt/bin/etcdctl mk /coreos.com/network/config '{"Network":"10.0.0.0/16"}'

    You can use the below command on another node to comfirm if the network setting is correct.

    $ /opt/bin/etcdctl get /coreos.com/network/config

    If you got {"Network":"10.0.0.0/16"}, then etcd cluster is working well. If not , please check/var/log/upstart/etcd.log to resolve etcd problem before going forward. Finally, use ifconfig to see if there is a new network interface named flannel0 coming up.

  3. On every minion node

    Make sure you have brctl installed on every minion, otherwise please run sudo apt-get install bridge-utils

    $ sudo ./reconfigureDocker.sh

    This will make the docker daemon aware of flannel network.

All done !

IV. Validation

You can use kubectl command to see if the newly created k8s is working correctly.

For example , $ kubectl get minions to see if you get all your minion nodes comming up.

Also you can run kubernetes guest-example to build a redis backend cluster on the k8s.

V. Trouble Shooting

Generally, what of this guide did is quite simple:

  1. Build and copy binaries and configuration files to proper dirctories on every node

  2. Configure etcd using IPs based on input from user

  3. Create and start flannel network

So, whenver you have problem, do not blame Kubernetes, check etcd configuration first

Please try:

  1. Check /var/log/upstart/etcd.log for suspicisous etcd log

  2. Check /etc/default/etcd, as we do not have much input validation, a right config should be like:

    ETCD_OPTS="-name infra1 -initial-advertise-peer-urls <http://ip_of_this_node:2380> -listen-peer-urls <http://ip_of_this_node:2380> -initial-cluster-token etcd-cluster-1 -initial-cluster infra1=<http://ip_of_this_node:2380>,infra2=<http://ip_of_another_node:2380>,infra3=<http://ip_of_another_node:2380> -initial-cluster-state new"
  3. Remove data-dir of etcd and run reconfigureDocker.shagain, the default path of data-dir is /infra*.etcd/

  4. You can also customize your own settings in /etc/default/{component_name} after configured success.

Kubernetes deployed on multiple ubuntu nodes的更多相关文章

  1. 【云计算】Docker云平台—Docker进阶

    Docker云平台系列共三讲,此为第二讲:Docker进阶 参考资料: 五个Docker监控工具的对比:http://www.open-open.com/lib/view/open1433897177 ...

  2. Connecting Elixir Nodes with libcluster, locally and on Kubernetes

    转自:https://www.poeticoding.com/connecting-elixir-nodes-with-libcluster-locally-and-on-kubernetes/ Tr ...

  3. ubuntu物理机上搭建Kubernetes集群 -- 准备

    准备工作 1.kubernetes架构 2.三台ubuntu主机: 操作系统:ubuntu-16.04.1-server-amd64 docker: 1.安装 使用命令 sudo apt-get in ...

  4. 在Ubuntu安装kubernetes

    一.安装Docker 1. 配置Docker docker安装完成后需要配置cgroup驱动为systemd来增强稳定性 sudo vim /etc/docker/daemon.json { &quo ...

  5. 101 More Security Best Practices for Kubernetes

    https://rancher.com/blog/2019/2019-01-17-101-more-kubernetes-security-best-practices/ The CNCF recen ...

  6. 离线环境下使用二进制方式安装配置Kubernetes集群

    本文环境 Redhat Linux 7.3,操作系统采用的最小安装方式. Kubernetes的版本为 V1.10. Docker版本为18.03.1-ce. etcd 版本为 V3.3.8. 1. ...

  7. Ubuntu 14.04主机上部署k8s集群

    部署结构 3台虚拟机,其中1台作为master,2台作为minion,都安装了最新版本的docker engine(目前是1.11.2) k8s版本是1.3.0 主要问题 部署步骤基本按照官方文档:h ...

  8. 基于Containerd安装部署高可用Kubernetes集群

    转载自:https://blog.weiyigeek.top/2021/7-30-623.html 简述 Kubernetes(后续简称k8s)是 Google(2014年6月) 开源的一个容器编排引 ...

  9. 在Azure Container Service创建Kubernetes(k8s)群集运行ASP.NET Core跨平台应用程序

    引子 在此前的一篇文章中,我介绍了如何在本地docker环境中运行ASP.NET Core跨平台应用程序(http://www.cnblogs.com/chenxizhang/p/7148657.ht ...

随机推荐

  1. iOS 多线程 浅述

    什么是进程? 进程是指在系统中正在运行的一个应用程序. 每个进程之间是独立的,每个进程均运行在其专用且受保护的内存空间内. 什么是线程? 1个进程要想执行任务,必须得有线程(每1个进程至少要有1条线程 ...

  2. 【CoreData】表之间的关联

    这次是表之间怎么进行关联,要求如下: // 建立学生与班级表之间的联系 既然是表与表之间的关联,那肯定是要先创建表: // 1.创建模型文件 (相当于一个数据库里的表) // New File ——— ...

  3. 关于OC中的block自己的一些理解(一)

    一.关于block 1.block的作用:保存一段代码. 2.苹果官方推荐的一种语法,类似于C语言的函数,但是比函数更加灵活. 3.^是block语法的标识. 二.block的用法 1)无返回值无参数 ...

  4. Android系统性能调优工具介绍

    http://blog.csdn.net/innost/article/details/9008691 经作者授权,发表Tieto某青年牛的一篇<程序员>大作. Android系统性能调优 ...

  5. Maven在Windows上的安装与配置

    下载Maven 1.进入Maven官网下载:maven.apache.org(点击进入或复制浏览器地址栏,回车进入) 注意事项:需要大家要注意的是(截至目前2015年12月1日)最新版本的Maven3 ...

  6. mac 远程连接服务器

    很多刚用mac的同学 可能会纠结,连接远程服务器咋整? 有没有类型windows上的securecrt 其实,完全可以不用: mac自带的终端就可以搞定:终端terminal 如何连接远程服务器? s ...

  7. XML语言基础1

    这学期选修了XML技术这门课,没有发课本,于是参考了W3school教程,整理一下上课的内容. 1.XML简介 XML是一种标记语言,很类似HTML,它不是对HTML的替代,而是对HTML的补充.在大 ...

  8. Sublime更换默认字体的方法

    Sublime是一款很不错的编辑器,不过默认安装后的字体却不尽人意,并且Sublime竟然连个完整的设置页面都没有(直接让你编辑配置文件).于是很多人对这字体就忍气吞声了.其实只要添加一行代码就可以完 ...

  9. android中的万能适配器BaseAdapter的总结

    有时候,列表不光会用来做显示用,我们同样可以在在上面添加按钮.添加按钮首先要写一个有按钮的xml文件,然后自然会想到用上面的方法定义一个适配器,然后将数据映射到布局文件上.但是事实并非这样,因为按钮是 ...

  10. JavaScript(三)——DOM操作一

    一.DOM的基本概念 DOM是文档对象模型,这种模型为树模型:文档是指标签文档:对象是指文档中每个元素:模型是指抽象化的东西. 二.Window对象操作 1.属性和方法: 属性(值或者子对象): op ...