Linux服务器上搭建codis集群之——安装前环境准备

codis是redis的分布式集群模式，由豌豆荚开源，本文简单记录一下它的集群搭建方法。

首先介绍一下我的实验环境。三台配置相同的虚拟机，

[root@test ~]# ip a|grep -w "inet"|grep -vw lo
    inet 10.0.0.10/ brd 10.0.0.255 scope global eth0
[root@test ~]# uname -r
2.6.-.el6.x86_64
[root@test ~]# cat /etc/redhat-release
CentOS release 6.10 (Final)
[root@test ~]# 

安装codis之前，我们需要准备的有，JAVA环境、GO环境、Zookeeper协调服务，codis安装包

第一步：

安装java环境，三台全部安装，因为都有运行zookeeper。jdk最好去官网下载一个rpm包安装，yum源里的openjdk运行时会有点小问题，为了减少不必要的问题，我们就不使用openjdk了。rpm包自己去下载就好了，下载之后传到服务器上。

[root@test1 ~]# rz -y
rz waiting to receive.
 zmodem trl+C ȡ

  %   KB  KB/s ::        Errors...

[root@test1 ~]# ll
total
-rw-r--r--.  root root  Nov     jdk-8u181-linux-x64.rpm
[root@test1 ~]# for host in  ;do scp jdk-8u181-linux-x64.rpm 10.0..$host:/root;done
root@10.0.0.11's password:
jdk-8u181-linux-x64.rpm                                                                            %  162MB  .2MB/s   :
root@10.0.0.12's password:
jdk-8u181-linux-x64.rpm
[root@test1 ~]# for host in   ;do ssh 10.0..$host rpm -ivh jdk-8u181-linux-x64.rpm ;done
The authenticity of host '10.0.0.10 (10.0.0.10)' can't be established.
RSA key fingerprint is f9:d9:c2:cb:cc:0d:ef:9e:ba:::a3:b0::1b:2f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.0.10' (RSA) to the list of known hosts.
root@10.0.0.10's password:
warning: jdk-8u181-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...                ##################################################
        package jdk1.-:1.8.0_181-fcs.x86_64 is already installed
root@10.0.0.11's password:
warning: jdk-8u181-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...                ##################################################
jdk1.                      ##################################################
Unpacking JAR files...
        tools.jar...
        plugin.jar...
        javaws.jar...
        deploy.jar...
        rt.jar...
        jsse.jar...
        charsets.jar...
        localedata.jar...
root@10.0.0.12's password:
warning: jdk-8u181-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Preparing...                ##################################################
jdk1.                      ##################################################
Unpacking JAR files...
        tools.jar...
        plugin.jar...
        javaws.jar...
        deploy.jar...
        rt.jar...
        jsse.jar...
        charsets.jar...
        localedata.jar...
[root@test1 ~]# 

安装完成之后验证是否成功安装。

[root@test1 ~]# java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) -Bit Server VM (build 25.181-b13, mixed mode)
[root@test1 ~]# 

第二步：

安装zookeeper，三台服务器全部安装。

下载zookeeper的安装包

[root@test1 ~]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/stable/zookeeper-3.4.14.tar.gz

[root@test1 ~]# ll
total 202836
-rw-r--r--. 1 root root 170023183 Nov 2 2018 jdk-8u181-linux-x64.rpm
-rw-r--r--. 1 root root 37676320 Apr 1 22:44 zookeeper-3.4.14.tar.gz

安装zookeeper，修改配置文件，把我们的zookeeper节点加入zookeeper集群中。

[root@test1 ~]# tar xf zookeeper-3.4..tar.gz -C /usr/local/
[root@test1 ~]# cd /usr/local/
[root@test1 local]# ls
bin etc games include lib lib64 libexec sbin share src zookeeper-3.4.
[root@test1 local]# mv zookeeper-3.4./ zookeeper
[root@test1 local]# cd zookeeper/

[root@test1 zookeeper]# cp conf/zoo_sample.cfg conf/zoo.cfg
最后修改后的结果是
[root@test1 conf]# grep -Ev "^#|^$" zoo.cfg
tickTime=
initLimit=
syncLimit=
dataDir=/data/zk/data　　　　　　　　　　#zookeeper的数据存放目录，我们要自己创建出来
clientPort=
server.=10.0.0.10::3888　　　　　　#这三个节点我的zookeeper节点
server.=10.0.0.11::
server.=10.0.0.12::

创建数据目录和myid标识文件

[root@test1 conf]# mkdir /data/zk/data -p　　　　#这个目录在三个节点上都要创建
[root@test1 conf]# echo  >/data/zk/data/myid　　#每个节点的myid数值要不相同

启动zookeeper，这里注意一下，要注意服务器的防火墙问题，防火墙会阻止集群节点之间的通信，导致集群无法建立。建议关闭防火墙。

三个节点都要启动
[root@test3 bin]# cd /usr/local/zookeeper/bin/
[root@test3 bin]# ./zkServer.sh start

查看集群的状态

[root@test3 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
这表明启动成功，并且该节点是集群的领导者

如果出现这种问题，说明你的服务没起来，查看一下服务器上是否启动了zookeeper的进程，如果进程存在，检查一下几台节点之间的防火墙状态，可能是防火墙阻挡了集群通信。

[root@test1 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
查看日志信息发现是防火墙的问题
[root@test1 bin]# tailf zookeeper.out
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:)
        at java.net.Socket.connect(Socket.java:)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, [myid:] - INFO  [QuorumPeer[myid=]/:::::::::QuorumPeer$QuorumServer@] - Resolved hostname: 10.0.0.12 to address: /10.0.0.12
-- ::, [myid:] - INFO  [QuorumPeer[myid=]/:::::::::FastLeaderElection@] - Notification time out:
-- ::, [myid:] - WARN  [QuorumPeer[myid=]/:::::::::QuorumCnxManager@] - Cannot open channel to  at election address /10.0.0.11:
java.net.NoRouteToHostException: No route to host (Host unreachable)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:)
        at java.net.Socket.connect(Socket.java:)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)

第三步

安装GO环境，这个环境只要在安装codis的服务器上配置就好了，我的是10.0.0.10这台机器。

[root@test2 bin]# yum install epel-release -y
[root@test2 bin]# yum install go -y

验证：

[root@test1 bin]# go version
go version go1.11.5 linux/amd64

到此，我们的准备工作就完成了，限于篇幅原因，下篇写codis安装。