zookeeper集群安装(转)
转载地址:http://www.blogjava.net/hello-yun/archive/2012/05/03/377250.html
本方法,本人亲自试验,可以成功。
ZooKeeper是一个分布式开源框架,提供了协调分布式应用的基本服务,它向外部应用暴露一组通用服务——分布式同步(Distributed Synchronization)、命名服务(Naming Service)、集群维护(Group Maintenance)等,简化分布式应用协调及其管理的难度,提供高性能的分布式服务。ZooKeeper本身可以以Standalone模式安装运行,不过它的长处在于通过分布式ZooKeeper集群(一个Leader,多个Follower),基于一定的策略来保证ZooKeeper集群的稳定性和可用性,从而实现分布式应用的可靠性。
有关ZooKeeper的介绍,网上很多,也可以参考文章后面,我整理的一些相关链接。
下面,我们简单说明一下ZooKeeper的配置。
1 ZooKeeper Standalone模式
从Apache网站上(zookeeper.apache.org)下载ZooKeeper软件包,我选择了3.3.4版本的(zookeeper-3.3.4.tar.gz),在一台Linux机器上安装非常容易,只需要解压缩后,简单配置一下即可以启动ZooKeeper服务器进程。
将zookeeper-3.3.4/conf目录下面的 zoo_sample.cfg修改为zoo.cfg,配置文件内容如下所示:
tickTime=2000
dataDir=/home/hadoop/storage/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
上面各个配置参数的含义也非常简单,引用如下所示:
tickTime —— the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime.
dataDir —— the location to store the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database.
clientPort —— the port to listen for client connections
下面启动ZooKeeper服务器进程:
cd zookeeper-3.3./
bin/zkServer.sh start
通过jps命令可以查看ZooKeeper服务器进程,名称为QuorumPeerMain。
在客户端连接ZooKeeper服务器,执行如下命令:
bin/zkCli.sh -server dynamic:
上面dynamic是我的主机名,如果在本机执行,则执行如下命令即可:
bin/zkCli.sh
客户端连接信息如下所示:
接着,可以使用help查看Zookeeper客户端可以使用的基本操作命令。
2 ZooKeeper Distributed模式
ZooKeeper分布式模式安装(ZooKeeper集群)也比较容易,这里说明一下基本要点。
首先要明确的是,ZooKeeper集群是一个独立的分布式协调服务集群,“独立”的含义就是说,如果想使用ZooKeeper实现分布式应用的协调与管理,简化协调与管理,任何分布式应用都可以使用,这就要归功于Zookeeper的数据模型(Data Model)和层次命名空间(Hierarchical Namespace)结构,详细可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html。在设计你的分布式应用协调服务时,首要的就是考虑如何组织层次命名空间。
下面说明分布式模式的安装配置,过程如下所示:
第一步:主机名称到IP地址映射配置
ZooKeeper集群中具有两个关键的角色:Leader和Follower。集群中所有的结点作为一个整体对分布式应用提供服务,集群中每个结点之间都互相连接,所以,在配置的ZooKeeper集群的时候,每一个结点的host到IP地址的映射都要配置上集群中其它结点的映射信息。
例如,我的ZooKeeper集群中每个结点的配置,以slave-01为例,/etc/hosts内容如下所示:
192.168.0.179 slave-
192.168.0.178 slave-
192.168.0.177 slave-
ZooKeeper采用一种称为Leader election的选举算法。在整个集群运行过程中,只有一个Leader,其他的都是Follower,如果ZooKeeper集群在运行过程中Leader出了问题,系统会采用该算法重新选出一个Leader。因此,各个结点之间要能够保证互相连接,必须配置上述映射。
ZooKeeper集群启动的时候,会首先选出一个Leader,在Leader election过程中,某一个满足选举算的结点就能成为Leader。整个集群的架构可以参考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html#sc_designGoals。
第二步:修改ZooKeeper配置文件
在其中一台机器(slave-01)上,解压缩zookeeper-3.3.4.tar.gz,修改配置文件conf/zoo.cfg,内容如下所示:
tickTime=
dataDir=/home/hadoop/storage/zookeeper
clientPort=
initLimit=
syncLimit=
server.=slave-::
server.=slave-::
server.=slave-::
上述配置内容说明,可以参考http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper。
第三步:远程复制分发安装文件
上面已经在一台机器slave-01上配置完成ZooKeeper,现在可以将该配置好的安装文件远程拷贝到集群中的各个结点对应的目录下:
cd /home/hadoop/installation/
scp -r zookeeper-3.3./ hadoop@slave-:/home/hadoop/installation/
scp -r zookeeper-3.3./ hadoop@slave-:/home/hadoop/installation/
第四步:设置myid
在我们配置的dataDir指定的目录下面,创建一个myid文件,里面内容为一个数字,用来标识当前主机,conf/zoo.cfg文件中配置的server.X中X为什么数字,则myid文件中就输入这个数字,例如:
hadoop@slave-:~/installation/zookeeper-3.3.$ echo "" > /home/hadoop/storage/zookeeper/myid
hadoop@slave-:~/installation/zookeeper-3.3.$ echo "" > /home/hadoop/storage/zookeeper/myid
hadoop@slave-:~/installation/zookeeper-3.3.$ echo "" > /home/hadoop/storage/zookeeper/myid
按照上述进行配置即可。
第五步:启动ZooKeeper集群
在ZooKeeper集群的每个结点上,执行启动ZooKeeper服务的脚本,如下所示:
hadoop@slave-01:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
hadoop@slave-02:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
hadoop@slave-03:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
以结点slave-01为例,日志如下所示:
hadoop@slave-:~/installation/zookeeper-3.3.$ tail -500f zookeeper.out
-- ::, - INFO [main:QuorumPeerConfig@] - Reading configuration from: /home/hadoop/installation/zookeeper-3.3./bin/../conf/zoo.cfg
-- ::, - INFO [main:QuorumPeerConfig@] - Defaulting to majority quorums
-- ::, - INFO [main:QuorumPeerMain@] - Starting quorum peer
-- ::, - INFO [main:NIOServerCnxn$Factory@] - binding to port 0.0.0.0/0.0.0.0:
-- ::, - INFO [main:QuorumPeer@] - tickTime set to
-- ::, - INFO [main:QuorumPeer@] - minSessionTimeout set to -
-- ::, - INFO [main:QuorumPeer@] - maxSessionTimeout set to -
-- ::, - INFO [main:QuorumPeer@] - initLimit set to
-- ::, - INFO [Thread-:QuorumCnxManager$Listener@] - My election bind port:
-- ::, - INFO [QuorumPeer:/:::::::::QuorumPeer@] - LOOKING
-- ::, - INFO [QuorumPeer:/:::::::::FastLeaderElection@] - New election. My id = , Proposed zxid =
-- ::, - INFO [WorkerReceiver Thread:FastLeaderElection@] - Notification: (n.leader), (n.zxid), (n.round), LOOKING (n.state), (n.sid), LOOKING (my state)
-- ::, - WARN [WorkerSender Thread:QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.178:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:)
at java.lang.Thread.run(Thread.java:)
-- ::, - WARN [WorkerSender Thread:QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.177:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:)
at java.lang.Thread.run(Thread.java:)
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.178:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.177:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - INFO [QuorumPeer:/:::::::::FastLeaderElection@] - Notification time out:
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.178:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.177:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - INFO [QuorumPeer:/:::::::::FastLeaderElection@] - Notification time out:
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.178:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.177:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - INFO [QuorumPeer:/:::::::::FastLeaderElection@] - Notification time out:
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.178:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.177:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - INFO [QuorumPeer:/:::::::::FastLeaderElection@] - Notification time out:
-- ::, - INFO [WorkerReceiver Thread:FastLeaderElection@] - Notification: (n.leader), (n.zxid), (n.round), LOOKING (n.state), (n.sid), LOOKING (my state)
-- ::, - INFO [QuorumPeer:/:::::::::FastLeaderElection@] - Updating proposal
-- ::, - WARN [WorkerSender Thread:QuorumCnxManager@] - Cannot open channel to at election address slave-/192.168.0.177:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:)
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:)
at java.lang.Thread.run(Thread.java:)
-- ::, - INFO [WorkerReceiver Thread:FastLeaderElection@] - Notification: (n.leader), (n.zxid), (n.round), LOOKING (n.state), (n.sid), LOOKING (my state)
-- ::, - INFO [QuorumPeer:/:::::::::QuorumPeer@] - FOLLOWING
-- ::, - INFO [QuorumPeer:/:::::::::Learner@] - TCP NoDelay set to: true
-- ::, - INFO [QuorumPeer:/:::::::::Environment@] - Server environment:zookeeper.version=3.3.-, built on // : GMT
-- ::, - INFO [QuorumPeer:/:::::::::Environment@] - Server environment:host.name=slave-
-- ::, - INFO [QuorumPeer:/:::::::::Environment@] - Server environment:java.version=1.6.0_30
-- ::, - INFO [QuorumPeer:/:::::::::Environment@] - Server environment:java.vendor=Sun Microsystems Inc.
-- ::, - INFO [QuorumPeer:/:::::::::Environment@] - Server environment:java.home=/home/hadoop/installation/jdk1..0_30/jre
-- ::, - INFO [QuorumPeer:/:::::::::Environment@] - Server environment:java.class.path=/home/hadoop/installation/zookeeper-3.3./bin/../build/classes:/home/hadoop/installation/zookeeper-3.3./bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2012-01-08 06:51:23,210 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2012-01-08 06:51:23,210 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.io.tmpdir=/tmp
2012-01-08 06:51:23,212 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.compiler=<NA>
2012-01-08 06:51:23,212 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.name=Linux
2012-01-08 06:51:23,212 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.arch=i386
2012-01-08 06:51:23,213 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.version=3.0.0-14-generic
2012-01-08 06:51:23,213 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.name=hadoop
2012-01-08 06:51:23,214 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.home=/home/hadoop
2012-01-08 06:51:23,214 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2012-01-08 06:51:23,223 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@151] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/hadoop/storage/zookeeper/version-2 snapdir /home/hadoop/storage/zookeeper/version-2
2012-01-08 06:51:23,339 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@294] - Getting a snapshot from leader
2012-01-08 06:51:23,358 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@325] - Setting leader epoch 1
2012-01-08 06:51:23,358 - INFO [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FileTxnSnapLog@254] - Snapshotting: 0
2012-01-08 06:51:25,511 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
2012-01-08 06:51:42,584 - INFO [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
我启动的顺序是slave-01>slave-02>slave-03,由于ZooKeeper集群启动的时候,每个结点都试图去连接集群中的其它结点,先启动的肯定连不上后面还没启动的,所以上面日志前面部分的异常是可以忽略的。通过后面部分可以看到,集群在选出一个Leader后,最后稳定了。
其他结点可能也出现类似问题,属于正常。
第六步:安装验证
可以通过ZooKeeper的脚本来查看启动状态,包括集群中各个结点的角色(或是Leader,或是Follower),如下所示,是在ZooKeeper集群中的每个结点上查询的结果:
hadoop@slave-:~/installation/zookeeper-3.3.$ bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3./bin/../conf/zoo.cfg
Mode: follower hadoop@slave-:~/installation/zookeeper-3.3.$ bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3./bin/../conf/zoo.cfg
Mode: leader hadoop@slave-:~/installation/zookeeper-3.3.$ bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3./bin/../conf/zoo.cfg
Mode: follower
通过上面状态查询结果可见,slave-02是集群的Leader,其余的两个结点是Follower。
另外,可以通过客户端脚本,连接到ZooKeeper集群上。对于客户端来说,ZooKeeper是一个整体(ensemble),连接到ZooKeeper集群实际上感觉在独享整个集群的服务,所以,你可以在任何一个结点上建立到服务集群的连接,例如:
hadoop@slave-:~/installation/zookeeper-3.3.$ bin/zkCli.sh -server slave-:
Connecting to slave-:
-- ::, - INFO [main:Environment@] - Client environment:zookeeper.version=3.3.-, built on // : GMT
-- ::, - INFO [main:Environment@] - Client environment:host.name=slave-
-- ::, - INFO [main:Environment@] - Client environment:java.version=1.6.0_30
-- ::, - INFO [main:Environment@] - Client environment:java.vendor=Sun Microsystems Inc.
-- ::, - INFO [main:Environment@] - Client environment:java.home=/home/hadoop/installation/jdk1..0_30/jre
-- ::, - INFO [main:Environment@] - Client environment:java.class.path=/home/hadoop/installation/zookeeper-3.3./bin/../build/classes:/home/hadoop/installation/zookeeper-3.3./bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2012-01-08 07:14:21,111 - INFO [main:Environment@97] - Client environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2012-01-08 07:14:21,116 - INFO [main:Environment@97] - Client environment:java.io.tmpdir=/tmp
2012-01-08 07:14:21,124 - INFO [main:Environment@97] - Client environment:java.compiler=<NA>
2012-01-08 07:14:21,169 - INFO [main:Environment@97] - Client environment:os.name=Linux
2012-01-08 07:14:21,175 - INFO [main:Environment@97] - Client environment:os.arch=i386
2012-01-08 07:14:21,177 - INFO [main:Environment@97] - Client environment:os.version=3.0.0-14-generic
2012-01-08 07:14:21,185 - INFO [main:Environment@97] - Client environment:user.name=hadoop
2012-01-08 07:14:21,188 - INFO [main:Environment@97] - Client environment:user.home=/home/hadoop
2012-01-08 07:14:21,190 - INFO [main:Environment@97] - Client environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2012-01-08 07:14:21,197 - INFO [main:ZooKeeper@379] - Initiating client connection, connectString=slave-01:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@bf32c
2012-01-08 07:14:21,305 - INFO [main-SendThread():ClientCnxn$SendThread@1061] - Opening socket connection to server slave-01/192.168.0.179:2181
Welcome to ZooKeeper!
2012-01-08 07:14:21,376 - INFO [main-SendThread(slave-01:2181):ClientCnxn$SendThread@950] - Socket connection established to slave-01/192.168.0.179:2181, initiating session
JLine support is enabled
[zk: slave-01:2181(CONNECTING) 0] 2012-01-08 07:14:21,872 - INFO [main-SendThread(slave-01:2181):ClientCnxn$SendThread@739] - Session establishment complete on server slave-01/192.168.0.179:2181, sessionid = 0x134bdcd6b730000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: slave-01:2181(CONNECTED) 0] ls /
[zookeeper]
当前根路径为/zookeeper。
总结说明
主机名与IP地址映射配置问题
启动ZooKeeper集群时,如果ZooKeeper集群中slave-01结点的日志出现如下错误:
java.net.SocketTimeoutException
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - INFO [QuorumPeer:/:::::::::FastLeaderElection@] - Notification time out:
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/202.106.199.35:
java.net.SocketTimeoutException
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:)
-- ::, - WARN [QuorumPeer:/:::::::::QuorumCnxManager@] - Cannot open channel to at election address slave-/202.106.199.35:
很显然,slave-01在启动时连接集群中其他结点(slave-02、slave-03)时,主机名映射的IP与我们实际配置的不一致,所以集群中各个结点之间无法建立链路,整个ZooKeeper集群启动是失败的。
上面错误日志中slave-02/202.106.199.35:3888实际应该是slave-02/202.192.168.0.178:3888就对了,但是在进行域名解析的时候映射有问题,修改每个结点的/etc/hosts文件,将ZooKeeper集群中所有结点主机名到IP地址的映射配置上。
zookeeper集群安装(转)的更多相关文章
- 1、zookeeper集群安装
前提准备3台centos7.0虚拟机 c7003:192.168.70.103 c7004:192.168.70.104 c7005:192.168.70.105 并在三台虚拟机上配置hosts为 1 ...
- 原创:centos7.1下 ZooKeeper 集群安装配置+Python实战范例
centos7.1下 ZooKeeper 集群安装配置+Python实战范例 下载:http://apache.fayea.com/zookeeper/zookeeper-3.4.9/zookeepe ...
- hbase和ZooKeeper集群安装配置
一:ZooKeeper集群安装配置 1:解压zookeeper-3.3.2.tar.gz并重命名为zookeeper. 2:进入~/zookeeper/conf目录: 拷贝zoo_sample.cfg ...
- Zookeeper集群安装Version3.5.1
Zookeeper集群安装,基于版本3.5.1, 使用zookeeper-3.5.1-alpha.tar.gz安装包. 1.安装规划 zookeeper集群模式,安装到如下三台机器 10.43.159 ...
- Zookeeper集群安装详解
Zookeeper的角色 Zookeeper集群搭建 要求:服务器集群规模不小于3个节点,各服务器之间系统时间要保持一致! 安装步骤 1.在h1节点解压,目录改名. tar –zxvf zooke ...
- ZooKeeper集群安装
ZooKeeper是Apache提供的.分布式服务协调系统,应用比較广泛. 由于项目中使用Kafka MQ,而Kafka全然使用ZooKeeper实现Kafka各组件的服务协调,包含Broker.Co ...
- Zookeeper 集群安装配置,超详细,速度收藏!
今天,栈长分享下 Zookeeper 的集群安装及配置. 下载 下载地址:http://zookeeper.apache.org/ 下载过程就不说了,我们下载了最新的zookeeper-3.4.11. ...
- centos7下zookeeper集群安装部署
应用场景:ZooKeeper是一个分布式的,开放源码的分布式应用程序协调服务,是Google的Chubby一个开源的实现,是Hadoop和Hbase的重要组件. 它是一个为分布式应用提供一致性服务的软 ...
- Zookeeper 集群安装配置
今天,栈长分享下 Zookeeper 的集群安装及配置. 下载 下载地址:http://zookeeper.apache.org/ 下载过程就不说了,我们下载了最新的zookeeper-3.4.11. ...
- zookeeper 集群安装与配置
Zookeeper安装和配置 准备工作 安装 JDK,此步略. 下载 zookeeper wget http://archive.apache.org/dist/zookeeper/zookeeper ...
随机推荐
- HTML实用案例(1)—— 左侧菜单,右侧内容的布局(带左侧菜单点击隐藏显示效果)
效果图 代码部分 <!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" ...
- 远程激活.NET REFLECTOR(不能断网)
由于是远程连接的服务器,使用.Net Reflector注册机激活时,不能断网. 如何禁止Reflector连接她的激活服务器,可进行如下操作: 1.添加hosts: 0.0.0.0 red-gate ...
- BZOJ3075,LG3082 [USACO13MAR]项链Necklace
题意 Bessie the cow has arranged a string of N rocks, each containing a single letter of the alphabet, ...
- 学习动态性能表(16)--v$rowcache
学习动态性能表 第16篇--V$ROWCACHE 2007.6.12 本视图显示数据字典缓存(也叫rowcache)的各项统计.每一条记录包含不同类型的数据字典缓存数据统计,注意数据字典缓存有层次差 ...
- python list和元祖
一,元祖 在python中元祖是只能查询和读取的一组数据,在()内的赋值就是元祖,只有查询和读取的功能: 1.len()方法:查询元祖有多少个元素 s = (') print(len(s)) 结果: ...
- AngularJS:表单
ylbtech-AngularJS:表单 1.返回顶部 1. AngularJS 表单 AngularJS 表单是输入控件的集合. HTML 控件 以下 HTML input 元素被称为 HTML 控 ...
- Git命令之创建版本
安装 安装好Git后,将会在桌面生成 这样一个图标 运行后将会是类似控制台程序的黑色窗口,其中mingw64(参考百度百科).这样的话就可以在输入命令 例如 :git 见到下图有详细的用法表示成功否则 ...
- PD中更改显示Name还是Code的设置
菜单->Tool->Model Options->Name Convention->右侧display中选择显示name还是code. 此外,在16版中,还可以通过Tool-D ...
- delphi 面向对象实用技能教学二(封装)
面向对象编程手法,是一项综合技能,单独把谁拿出来说都不合适.本次重写 TSimpleThread ,使其能在 D7 下运行. 基于 TSimpleThread ,重磅推出 TSimpleUI.ExeP ...
- Solaris Tips: Repairing the Boot Archive (ZT)
http://www.seedsofgenius.net/solaris/solaris-tips-repairing-the-boot-archive 注意以下是系统盘非镜像情况下的操作,如果系统盘 ...