Zookeeper环境搭建

　　zookeeper支持windows、linux、mac等操作系统，其搭建方式也有集群、伪集群、单机环境。下面研究三种方式的搭建。

　　单机环境:windows操作系统

　　伪集群:windows

　　集群:linux

1.单机环境

　　下面在windows下面搭建zookeeper的单机环境。windows下面也适合做开发。但是不适合生产环境的部署安装。

1.Java安装

　　这个就不说了，zookeeper依赖于Java环境，所以先自行安装JDK。

2.zookeeper下载以及安装

(1)下载zookeeperxxx.tar.gz

　　下载地址:http://zookeeper.apache.org/releases.html  下载xxx.tar.gz解压即可，里面有sh脚本，也有cmd脚本。

　　　　

(2)解压上面下载的文件

(3)在conf目录下面创建zoo.cfg (在conf目录下面提供了一分样本文件zoo_sample.cfg),或者复制一分样本文件改名为zoo.cfg

tickTime = 2000
dataDir = E:/zookeeper/zookeeper-3.4.13/data
clientPort = 2181
initLimit = 5
syncLimit = 2

解释:

　　tickTime：这个时间是作为 Zookeeper 服务器之间或客户端与服务器之间维持心跳的时间间隔，也就是每个 tickTime 时间就会发送一个心跳。
　　dataDir：顾名思义就是 Zookeeper 保存数据的目录，默认情况下，Zookeeper 将写数据的日志文件也保存在这个目录里。
　　dataLogDir：log目录, 同样可以是任意目录. 如果没有设置该参数, 将使用和dataDir相同的设置.
　　clientPort：这个端口就是客户端连接 Zookeeper 服务器的端口，Zookeeper 会监听这个端口，接受客户端的访问请求。

　　我在自己设置的时候同事配置dataDir和dataLogDir反而启动报错，所以我只设置了dataDir

(4)在对应目录建立上面的data目录即可

(5)启动zookeeper

E:\zookeeper\zookeeper-3.4.\bin> .\zkServer.cmd

E:\zookeeper\zookeeper-3.4.\bin>call "C:\Program Files\Java\jdk1.8.0_121"\bin\java "-Dzookeeper.log.dir=E:\zookeeper\zookeeper-3.4.13\bin\.." "-Dzookeeper.root.logger=INFO,CONSOLE" -cp "E:\zookeeper\zookeeper-3.4.13\bin\..\build\classes;E:\zookeeper\zookeeper-3.4.13\bin\..\build\lib\*;E:\zookeeper\zookeeper-3.4.13\bin\..\*;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\*;E:\zookeeper\zookeeper-3.4.13\bin\..\conf" org.apache.zookeeper.server.quorum.QuorumPeerMain "E:\zookeeper\zookeeper-3.4.13\bin\..\conf\zoo.cfg"
-- ::, [myid:] - INFO  [main:QuorumPeerConfig@] - Reading configuration from: E:\zookeeper\zookeeper-3.4.\bin\..\conf\zoo.cfg
-- ::, [myid:] - INFO  [main:DatadirCleanupManager@] - autopurge.snapRetainCount set to
-- ::, [myid:] - INFO  [main:DatadirCleanupManager@] - autopurge.purgeInterval set to
-- ::, [myid:] - INFO  [main:DatadirCleanupManager@] - Purge task is not scheduled.
-- ::, [myid:] - WARN  [main:QuorumPeerMain@] - Either no config or no quorum defined in config, running  in standalone mode
-- ::, [myid:] - INFO  [main:QuorumPeerConfig@] - Reading configuration from: E:\zookeeper\zookeeper-3.4.\bin\..\conf\zoo.cfg
-- ::, [myid:] - INFO  [main:ZooKeeperServerMain@] - Starting server
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:zookeeper.version=3.4.-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on // : GMT
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:host.name=MicroWin10-
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:java.version=1.8.0_121
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:java.vendor=Oracle Corporation
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:java.home=C:\Program Files\Java\jdk1..0_121\jre
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:java.class.path=E:\zookeeper\zookeeper-3.4.\bin\..\build\classes;E:\zookeeper\zookeeper-3.4.\bin\..\build\lib\*;E:\zookeeper\zookeeper-3.4.\bin\..\zookeeper-3.4..jar;E:\zookeeper\zookeeper-3.4.\bin\..\lib\audience-annotations-0.5..jar;E:\zookeeper\zookeeper-3.4.\bin\..\lib\jline-0.9..jar;E:\zookeeper\zookeeper-3.4.\bin\..\lib\log4j-1.2..jar;E:\zookeeper\zookeeper-3.4.\bin\..\lib\netty-3.10..Final.jar;E:\zookeeper\zookeeper-3.4.\bin\..\lib\slf4j-api-1.7..jar;E:\zookeeper\zookeeper-3.4.\bin\..\lib\slf4j-log4j12-1.7..jar;E:\zookeeper\zookeeper-3.4.\bin\..\conf
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:java.library.path=C:\Program Files\Java\jdk1..0_121\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;E:\Anaconda3\exe;E:\Anaconda3\exe\Library\mingw-w64\bin;E:\Anaconda3\exe\Library\usr\bin;E:\Anaconda3\exe\Library\bin;E:\Anaconda3\exe\Scripts;E:\ImageMagick-7.0.-Q16;C:\oraclexe\app\oracle\product\11.2.\server\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.\;E:\soft\apache-maven-3.5.\bin;C:\WINDOWS\System32\OpenSSH\;E:\git\Git\cmd;E:\SVN\bin;C:\Program Files (x86)\Microsoft SQL Server\\Tools\Binn\;C:\Program Files\Microsoft SQL Server\\Tools\Binn\;C:\Program Files\Microsoft SQL Server\\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\;C:\Program Files (x86)\Microsoft SQL Server\\DTS\Binn\;C:\Users\Administrator\AppData\Local\Microsoft\WindowsApps;E:\soft\maven\apache-maven-3.3.\bin;C:\Program Files\MySQL\MySQL Server 5.7\bin;C:\Program Files\Java\jdk1..0_121\bin;E:\git\Git\bin;E:\git\Git\usr\bin;E:\git\Git;C:\Program Files\Java\jdk1..0_121\jre\bin;D:\zdcontomcat\zdc8\lo\program;E:\tesseract4\Tesseract-OCR;E:\redis\redis-win;;.
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:java.io.tmpdir=C:\Users\ADMINI~\AppData\Local\Temp\
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:java.compiler=<NA>
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:os.name=Windows
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:os.arch=amd64
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:os.version=10.0
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:user.name=Administrator
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:user.home=C:\Users\Administrator
-- ::, [myid:] - INFO  [main:Environment@] - Server environment:user.dir=E:\zookeeper\zookeeper-3.4.\bin
-- ::, [myid:] - INFO  [main:ZooKeeperServer@] - tickTime set to
-- ::, [myid:] - INFO  [main:ZooKeeperServer@] - minSessionTimeout set to -
-- ::, [myid:] - INFO  [main:ZooKeeperServer@] - maxSessionTimeout set to -
-- ::, [myid:] - INFO  [main:ServerCnxnFactory@] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
-- ::, [myid:] - INFO  [main:NIOServerCnxnFactory@] - binding to port 0.0.0.0/0.0.0.0:

3.测试

(1)用netstat或者jps查看都可以

C:\Users\Administrator>netstat -ano|findstr
  TCP    0.0.0.0:           0.0.0.0:              LISTENING
  TCP    [::]:              [::]:                 LISTENING       

C:\Users\Administrator>jps -l
 sun.tools.jps.Jps
 org.apache.zookeeper.server.quorum.QuorumPeerMain

(2)zkCli.cmd连接进行测试:

PS E:\zookeeper\zookeeper-3.4.\bin> .\zkCli.cmd
Connecting to localhost:
.........
[zk: localhost:(CONNECTING) ] -- ::, [myid:] - INFO  [main-SendThread(localhost:):ClientCnxn$SendThread@] - Session establishment complete on server localhost/::::::::, sessionid = 0x10035685eb40002, negotiated timeout = 

WATCHER::

WatchedEvent state:SyncConnected type:None path:null

[zk: localhost:(CONNECTED) ]

[zk: localhost:(CONNECTED) ] help
ZooKeeper -server host:port cmd args
        stat path [watch]
        set path data [version]
        ls path [watch]
        delquota [-n|-b] path
        ls2 path [watch]
        setAcl path acl
        setquota -n|-b val path
        history
        redo cmdno
        printwatches on|off
        delete path [version]
        sync path
        listquota path
        rmr path
        get path [watch]
        create [-s] [-e] path data acl
        addauth scheme auth
        quit
        getAcl path
        close
        connect host:port
[zk: localhost:(CONNECTED) ] quit
Quitting...
-- ::, [myid:] - INFO  [main:ZooKeeper@] - Session: 0x10035685eb40002 closed
-- ::, [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@] - EventThread shut down for session: 0x10035685eb40002
PS E:\zookeeper\zookeeper-3.4.\bin>

2.伪集群安装-windows安装

　　伪集群，就是一台机器开启多个端口然后进行集群配置，最常用的学习方式。。。这个集群有点类似于ActiveMQ的伪集群。

1.将E:\zookeeper\zookeeper-3.4.13\conf\zoo.cfg复制出三分，分别命名为zoo1.cfg，zoo2.cfg，zoo3.cfg

对应内容:

zoo1.cfg

tickTime = 2000
dataDir = E:/zookeeper/zookeeper-3.4.13/data1
clientPort = 2181
initLimit = 5
syncLimit = 2

server.1=localhost:2887:3887
server.2=localhost:2888:3888
server.3=localhost:2889:3889

zoo2.cfg

tickTime = 2000
dataDir = E:/zookeeper/zookeeper-3.4.13/data2
clientPort = 2182
initLimit = 5
syncLimit = 2

server.1=localhost:2887:3887
server.2=localhost:2888:3888
server.3=localhost:2889:3889

zoo3.cfg

tickTime = 2000
dataDir = E:/zookeeper/zookeeper-3.4.13/data3
clientPort = 2183
initLimit = 5
syncLimit = 2

server.1=localhost:2887:3887
server.2=localhost:2888:3888
server.3=localhost:2889:3889

解释：    server.{num}=ip/domain:Port1:Port2

　　num：表示数字表示第几号服务器；

　　ip/domain ：是服务器域名或者ip地址。

　　 Port1：表示这个服务器和集群中的Leader服务器交换信息的端口；

　　 Port2：表示万一集群中的Leader服务器挂了，需要一个端口重新进行选举，选出一个新的Leader，这个端口就是用来执行选举时服务器相互通信的端口。

由于我们是伪集群，所以ip或者域名是一样的，所以要分配不同的端口号

2.创建对应的目录

3.进入E:\zookeeper\zookeeper-3.4.13\bin下复制文件zkServer.cmd为zkServer-1.cmd,zkServer-2.cmd,zkServer-3.cmd

zkServer-1.cmd内容修改为如下：

@echo off

setlocal
call "%~dp0zkEnv.cmd"

set ZOOCFG=E:/zookeeper/zookeeper-3.4.13/conf/zoo1.cfg
set ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain
echo on
call %JAVA% "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*

endlocal

zkServer-2.cmd内容修改为如下：

@echo off

setlocal
call "%~dp0zkEnv.cmd"

set ZOOCFG=E:/zookeeper/zookeeper-3.4.13/conf/zoo2.cfg
set ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain
echo on
call %JAVA% "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*

endlocal

zkServer-3.cmd内容修改为如下：

@echo off

setlocal
call "%~dp0zkEnv.cmd"

set ZOOCFG=E:/zookeeper/zookeeper-3.4.13/conf/zoo3.cfg
set ZOOMAIN=org.apache.zookeeper.server.quorum.QuorumPeerMain
echo on
call %JAVA% "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*

endlocal

4.添加myid文件

　　除了修改 zoo.cfg 配置文件，集群模式下还要配置一个标识自己身份也就是自己的ID值文件 myid，这个文件在zoo.cfg里dataDir指定的目录下，这个文件里面就只有一个数字，这个数字和server.n的n保持一致，该值范围可以是1-255之间，Zookeeper 启动时会读取这个文件，拿到里面的数据与 zoo.cfg 里面的配置信息比较从而判断到底是那个 server。

$ pwd
/e/zookeeper/zookeeper-3.4./data1

Administrator@MicroWin10- MINGW64 /e/zookeeper/zookeeper-3.4./data1
$ echo "">myid

Administrator@MicroWin10- MINGW64 /e/zookeeper/zookee                            per-3.4./data2
$ pwd
/e/zookeeper/zookeeper-3.4./data2

Administrator@MicroWin10- MINGW64 /e/zookeeper/zookeeper-3.4./data2
$ echo "">myid

Administrator@MicroWin10- MINGW64 /e/zookeeper/zookeeper-3.4./data3
$ echo "">myid

5.启动即可

PS E:\zookeeper\zookeeper-3.4.\bin> .\zkServer3.cmd

E:\zookeeper\zookeeper-3.4.\bin>call "C:\Program Files\Java\jdk1.8.0_121"\bin\java "-Dzookeeper.log.dir=E:\zookeeper\zookeeper-3.4.13\bin\.." "-Dzookeeper.root.logger=INFO,CONSOLE" -cp "E:\zookeeper\zookeeper-3.4.13\bin\..\build\classes;E:\zookeeper\zookeeper-3.4.13\bin\..\build\lib\*;E:\zookeeper\zookeeper-3.4.13\bin\..\*;E:\zookeeper\zookeeper-3.4.13\bin\..\lib\*;E:\zookeeper\zookeeper-3.4.13\bin\..\conf" org.apache.zookeeper.server.quorum.QuorumPeerMain "E:/zookeeper/zookeeper-3.4.13/conf/zoo3.cfg"
。。。。。。
-- ::, [myid:] - INFO  [QuorumPeer[myid=]/:::::::::ZooKeeperServer@] - Created server with tickTime  minSessionTimeout  maxSessionTimeout  datadir E:\zookeeper\zookeeper-3.4.\data3\version- snapdir E:\zookeeper\zookeeper-3.4.\data3\version-
-- ::, [myid:] - INFO  [QuorumPeer[myid=]/:::::::::Follower@] - FOLLOWING - LEADER ELECTION TOOK -
-- ::, [myid:] - INFO  [QuorumPeer[myid=]/:::::::::QuorumPeer$QuorumServer@] - Resolved hostname: localhost to address: localhost/127.0.0.1
-- ::, [myid:] - INFO  [QuorumPeer[myid=]/:::::::::Learner@] - Getting a snapshot from leader 0x100000000
-- ::, [myid:] - INFO  [QuorumPeer[myid=]/:::::::::FileTxnSnapLog@] - Snapshotting: 0x100000000 to E:\zookeeper\zookeeper-3.4.\data3\version-\snapshot.
-- ::, [myid:] - WARN  [QuorumPeer[myid=]/:::::::::Follower@] - Got zxid 0x100000001 expected 0x1
-- ::, [myid:] - INFO  [SyncThread::FileTxnLog@] - Creating new log file: log.

　　启动前两个服务器的时候报错链接拒绝，因为在通过端口选举leader的时候没找到端口。在console也看到一些关于选举leader等信息。

6.查看启动状态:

(1)jps查看

C:\Users\Administrator>jps
 QuorumPeerMain
 QuorumPeerMain
 QuorumPeerMain
 Jps

(2)连接集群的1281节点并创建第一条数据从1282节点查看数据

 .\zkCli.cmd -server 127.0.0.1:

创建节点:

[zk: 127.0.0.1:(CONNECTED) ] create /FirstZnode "Myfirstzookeeper-app"
Created /FirstZnode
[zk: 127.0.0.1:(CONNECTED) ] ls
[zk: 127.0.0.1:(CONNECTED) ] ls /
[zookeeper, FirstZnode]
[zk: 127.0.0.1:(CONNECTED) ] get /FirstZnode
Myfirstzookeeper-app
cZxid = 0x100000003
ctime = Thu Mar  :: CST
mZxid = 0x100000003
mtime = Thu Mar  :: CST
pZxid = 0x100000003
cversion =
dataVersion =
aclVersion =
ephemeralOwner = 0x0
dataLength =
numChildren = 

　　接下来链接到1282并继续查看 节点信息同上证明集群搭建成功。

(3)linux下面有zkServer.sh status查看节点状态，但是cmd脚本不支持传入参数，这个在研究源码之后编写的bat脚本可以实现windows下面查看zookeeper集群节点状态:

　　通过研究sh的源码发现其进行查询状态的时候是启动的org.apache.zookeeper.client.FourLetterWordMain类，我们查看org.apache.zookeeper.client.FourLetterWordMain的main函数需要的参数如下:

    public static void main(String[] args)
            throws IOException
    {
        if (args.length != 3) {
            System.out.println("Usage: FourLetterWordMain <host> <port> <cmd>");
        } else {
            System.out.println(send4LetterWord(args[0], Integer.parseInt(args[1]), args[2]));
        }
    }

　　所以我们自己写java命令也可以查看状态了。

于是自己编写的查看1281端口的bat脚本如下:  主要做的就是   清空classpath的值。然后重新赋值classpath的值，赋值classpath是为了引入zookeeper依赖的jar包

status1281.bat (需要放置到E:\zookeeper\zookeeper-3.4.13，也就是放置到与zookeeperxxx.jar同级目录)

e:
cd zookeeper\zookeeper-3.4.
set classpath=
set classpath=./*;./lib/*;
java -cp zookeeper-3.4.13.jar -classpath %classpath% org.apache.zookeeper.client.FourLetterWordMain 127.0.0.1 2181 status
pause

结果:

我们修改上面bat里面的脚本查看2182和2183的状态如下:

3.集群版

　　这里研究正式linux环境集群，使用虚拟机进行配置。

　　操作工具我们使用SecureCRT，方便同时向linux发送相同的命令。关于如何开启发送相同命令如下:

一、首先在SecureCRT里同时打开多个服务器session
二、选择菜单栏View -->Chat Windows  对号，此时所有服务器连接下方应该有个空白的部分
三、在空白的部分（Chat Windows）右键鼠标, 选上Send Chat to All Tabs, 这样Chat Windows里会有"<Send chat to all tabs>"的标志

0.当然首先在三个机器装JDK环境

1.准备三台centos操作系统，ip分别为:

　  　192.168.1.130       我们称为A机器

　　  192.168.1.131 　　我们称为B机器

　　  192.168.1.133　　我们称为C机器

2.建立相同的目录并上传zookeeperxxx.tar.gz

3.解压上面的目录 (三个做相同操作)

4.复制conf\zoo_sample.cfg到con\zoo.cfg目录下:

[root@localhost conf]# cp ./zoo_sample.cfg ./zoo.cfg
[root@localhost conf]# ls
configuration.xsl  log4j.properties  zoo.cfg  zoo_sample.cfg

5.修改 zoo.cfg

 A\B\C机器的内容都如下:

tickTime =
dataDir = /opt/zookeeper/zookeeper-3.4./data
clientPort =
initLimit =
syncLimit = 

server.=192.168.1.130::
server.=192.168.1.131::
server.=192.168.1.133::

6.同时创建三个服务器对应的数据目录

mkdir /opt/zookeeper/zookeeper-3.4./data

7.在三个机器的对应的data目录下面分别创建对应的id文件(重要)

A机器:

[root@localhost data]# echo "">myid
[root@localhost data]# cat myid

B机器:

[root@localhost data]# echo "">myid
[root@localhost data]# cat myid

C机器:

[root@localhost data]# echo "">myid
[root@localhost data]# cat myid

8.接下来启动三个服务器即可(同时发送命令)

/opt/zookeeper/zookeeper-3.4./bin/zkServer.sh start

9.JPS查看JVM

　　三个服务器都有主类为QuorumPeerMain的PID即启动成功。

10查看集群状态

/opt/zookeeper/zookeeper-3.4./bin/zkServer.sh status

A机器:

B机器:

C机器:

　　A\B为follower，C为leader。

注意:

(1)在上面必须有设置myid那一步，之前我好像没有那一步就启动报错

(2)我在上面查看的时候报错没有路由可以连接到主机，所以我猜想是防火墙的原因。我采用下面方式将三个主机的防火墙都关闭之后可以正常启动且查看状态。

service iptables stop