大数据技术之Hadoop3.1.2版本HA模式
大数据技术之Hadoop3.1.2版本HA模式
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.Hadoop的HA特点
1>.主备NameNode
2>.解决单点故障（属性，位置）
.主NameNode对外提供服务，备NameNode同步主NameNode元数据一带切换； .所有DataNode同时向两个NameNode汇报数据块信息（位置）； .JNN：集群（属性）； .Standby：备，完成了edits.log文件的合并产生新的image，推送回ANN；
3>.两种切换选择
.手动切换
通过命令实现主备之间的切换，可以用HDFS升级等场合。 .自动切换
基于Zookeeper实现。
4>.基于zookeeper自动切换方案
.Zookeeper Failover Controller进程用于监控NameNode监控状态； .并向zookeeper注册NameNode； .NameNode挂掉后，ZKFC为NameNode竞争锁，获得ZKFC锁的NameNode变为active；
5>.配置Hadoop3.1.2的官方文档（http://hadoop.apache.org/docs/r3.1.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html）
二.zookeeper完全分布式集群搭建
1>.下载zookeeper软件（下载地址：https://www.apache.org/dyn/closer.cgi/zookeeper/）
[root@node101.yinzhengjie.org.cn ~]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
---- ::-- https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
Resolving mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)... 101.6.8.193, :f000:::::
Connecting to mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)|101.6.8.193|:... connected.
HTTP request sent, awaiting response... OK
Length: (36M) [application/x-gzip]
Saving to: ‘zookeeper-3.4..tar.gz’ %[============================================================================================================================================>] ,, .57MB/s in .5s -- :: (7.94 MB/s) - ‘zookeeper-3.4..tar.gz’ saved [/] [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
2>.加压zookeeper并配置环境变量
[root@node101.yinzhengjie.org.cn ~]# tar -zxf zookeeper-3.4..tar.gz -C /yinzhengjie/softwares/
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/zookeeper-3.4./
total
drwxr-xr-x Mar : bin
-rw-rw-r-- Mar : build.xml
drwxr-xr-x Mar : conf
drwxr-xr-x Mar : dist-maven
-rw-rw-r-- Mar : ivysettings.xml
-rw-rw-r-- Mar : ivy.xml
drwxr-xr-x Mar : lib
-rw-rw-r-- Mar : LICENSE.txt
-rw-rw-r-- Mar : NOTICE.txt
-rw-rw-r-- Mar : pom.xml
-rw-rw-r-- Mar : README.md
-rw-rw-r-- Mar : README_packaging.txt
drwxr-xr-x Mar : src
-rw-rw-r-- Mar : zookeeper-3.4..jar
-rw-rw-r-- Mar : zookeeper-3.4..jar.asc
-rw-rw-r-- Mar : zookeeper-3.4..jar.md5
-rw-rw-r-- Mar : zookeeper-3.4..jar.sha1
drwxr-xr-x Mar : zookeeper-client
drwxr-xr-x Mar : zookeeper-contrib
drwxr-xr-x Mar : zookeeper-docs
drwxr-xr-x Mar : zookeeper-it
drwxr-xr-x Mar : zookeeper-jute
drwxr-xr-x Mar : zookeeper-recipes
drwxr-xr-x Mar : zookeeper-server
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# tar -zxf zookeeper-3.4.14.tar.gz -C /yinzhengjie/softwares/
[root@node101.yinzhengjie.org.cn ~]# tail - /etc/profile
#ADD ZOOKEEPER_HOME PATH
ZOOKEEPER_HOME=/yinzhengjie/softwares/zookeeper-3.4.
PATH=$PATH:$ZOOKEEPER_HOME/bin
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# source /etc/profile
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# tail -3 /etc/profile
3>.配置zookeeper的配置文件
[root@node101.yinzhengjie.org.cn ~]# cp /yinzhengjie/softwares/zookeeper-3.4./conf/zoo_sample.cfg /yinzhengjie/softwares/zookeeper-3.4./conf/zoo.cfg
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/zookeeper-3.4./conf/
total
-rw-rw-r-- Mar : configuration.xsl
-rw-rw-r-- Mar : log4j.properties
-rw-r--r-- root root Apr : zoo.cfg
-rw-rw-r-- Mar : zoo_sample.cfg
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cp /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo_sample.cfg /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo.cfg
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4./conf/zoo.cfg
# 滴答，计时的基本单位，默认是2000毫秒，即2秒。它是zookeeper最小的时间单位，用于丈量心跳时间和超时时间等，通常设置成默认2秒即可。
tickTime= # 初始化限制是10滴答，默认是10个滴答，即默认是20秒。指定follower节点初始化是链接leader节点的最大tick次数。
initLimit= # 数据同步的时间限制，默认是5个滴答，即默认时间是10秒。设定了follower节点与leader节点进行同步的最大时间。与initLimit类似，它也是以tickTime为单位进行指定的。
syncLimit= # 指定zookeeper的工作目录，这是一个非常重要的参数，zookeeper会在内存中在内存只能中保存系统快照，并定期写入该路径指定的文件夹中。生产环境中需要注意该文件夹的磁盘占用情况。
dataDir=/home/yinzhengjie/zookeeper # 监听zookeeper的默认端口。zookeeper监听客户端链接的端口，一般设置成默认2181即可。
clientPort= # 这个操作将限制连接到 ZooKeeper 的客户端的数量，限制并发连接的数量，它通过 IP 来区分不同的客户端。此配置选项可以用来阻止某些类别的 Dos 攻击。将它设置为 或者忽略而不进行设置将会取消对并发连接的限制。
#maxClientCnxns= # 在上文中已经提到，3.4.0及之后版本，ZK提供了自动清理事务日志和快照文件的功能，这个参数指定了清理频率，单位是小时，需要配置一个1或更大的整数，默认是0，表示不开启自动清理功能。
#autopurge.purgeInterval= # 这个参数和上面的参数搭配使用，这个参数指定了需要保留的文件数目。默认是保留3个。
#autopurge.snapRetainCount= #server.x=[hostname]:nnnnn[:nnnnn]，这里的x是一个数字，与myid文件中的id是一致的。右边可以配置两个端口，第一个端口用于F和L之间的数据同步和其它通信，第二个端口用于Leader选举过程中投票通信。
server.=node101.yinzhengjie.org.cn::
server.=node102.yinzhengjie.org.cn::
server.=node103.yinzhengjie.org.cn::
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo.cfg
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4./conf/java.env
#!/bin/bash
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie
#EMAIL:y1053419035@qq.com #指定JDK的安装路径
export JAVA_HOME=/yinzhengjie/softwares/jdk #指定zookeeper的heap内存大小
export JVMFLAGS="-Xms256m -Xmx256m $JVMFLAGS"
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4.14/conf/java.env #创建配置zookeeper的堆内存配置文件
4>.编写zookeeper的启动脚本
[root@node101.yinzhengjie.org.cn ~]# vi /usr/local/bin/xzk.sh
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /usr/local/bin/xzk.sh
#!/bin/bash
#@author :yinzhengjie
#blog:http://www.cnblogs.com/yinzhengjie
#EMAIL:y1053419035@qq.com #判断用户是否传参
if [ $# -ne ];then
echo "无效参数，用法为: $0 {start|stop|restart|status}"
exit
fi #获取用户输入的命令
cmd=$ #定义函数功能
function zookeeperManger(){
case $cmd in
start)
echo "启动服务"
remoteExecution start
;;
stop)
echo "停止服务"
remoteExecution stop
;;
restart)
echo "重启服务"
remoteExecution restart
;;
status)
echo "查看状态"
remoteExecution status
;;
*)
echo "无效参数，用法为: $0 {start|stop|restart|status}"
;;
esac
} #定义执行的命令
function remoteExecution(){
for (( i= ; i<= ; i++ )) ; do
tput setaf
echo ========== node${i}.yinzhengjie.org.cn zkServer.sh $ ================
tput setaf
ssh node${i}.yinzhengjie.org.cn "source /etc/profile ; zkServer.sh $1"
done
} #调用函数
zookeeperManger
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# vi /usr/local/bin/xzk.sh
[root@node101.yinzhengjie.org.cn ~]# ll /usr/local/bin/xzk.sh
-rw-r--r-- root root Apr : /usr/local/bin/xzk.sh
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# chmod +x /usr/local/bin/xzk.sh
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /usr/local/bin/xzk.sh
-rwxr-xr-x root root Apr : /usr/local/bin/xzk.sh
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# chmod +x /usr/local/bin/xzk.sh
5>.创建myid文件并写入服务器编号
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'mkdir /home/yinzhengjie/zookeeper/'
[WARNING]: Consider using file module with state=directory rather than running mkdir node101.yinzhengjie.org.cn | SUCCESS | rc= >> node103.yinzhengjie.org.cn | SUCCESS | rc= >> node102.yinzhengjie.org.cn | SUCCESS | rc= >> [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'mkdir /home/yinzhengjie/zookeeper/'
[root@node101.yinzhengjie.org.cn ~]# for (( i=;i<=;i++ )) do ssh node${i}.yinzhengjie.org.cn "echo -n $i > /home/yinzhengjie/zookeeper/myid" ;done
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'cat /home/yinzhengjie/zookeeper/myid'
node103.yinzhengjie.org.cn | SUCCESS | rc= >>
node102.yinzhengjie.org.cn | SUCCESS | rc= >>
node101.yinzhengjie.org.cn | SUCCESS | rc= >>
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# for (( i=101;i<=103;i++ )) do ssh node${i}.yinzhengjie.org.cn "echo -n $i > /home/yinzhengjie/zookeeper/myid" ;done
6>.分发配置文件
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node102.yinzhengjie.org.cn:/etc/
profile % .2MB/s :
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node102.yinzhengjie.org.cn:/etc/
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node103.yinzhengjie.org.cn:/etc/
profile % .4MB/s :
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node103.yinzhengjie.org.cn:/etc/
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4./ node102.yinzhengjie.org.cn:/yinzhengjie/softwares/
........
log. % .3KB/s :
log.63b % 48KB .7MB/s :
snapshot. % .6KB/s :
snapshot. % 55KB .5MB/s :
snapshot. % 55KB .3MB/s :
snapshot. % 140KB .2MB/s :
snapshot.83f % .5MB/s :
minikdc-krb5.conf % .3MB/s :
minikdc.ldiff % .2MB/s :
log. % 2401KB .2MB/s :
log.100001bf0 % 1001KB .6MB/s :
snapshot. % .1KB/s :
snapshot.100001bec % 2507KB .6MB/s :
check_compatibility.py % .7MB/s :
checkstyle-noframes-sorted.xsl % .5MB/s :
checkstyle.xml % .3MB/s :
findbugsExcludeFile.xml % .6MB/s :
log4j.properties % .9MB/s :
test-patch.properties % .7KB/s :
test-github-pr.sh % 24KB .7MB/s :
test-patch.sh % 24KB .4MB/s :
pom.xml % 10KB .6MB/s :
LICENSE.txt % 12KB .9MB/s :
NOTICE.txt % .2MB/s :
README.md % .4MB/s :
README_packaging.txt % .5MB/s :
build.xml % 95KB .5MB/s :
ivy.xml % 10KB .2MB/s :
ivysettings.xml % .3MB/s :
pom.xml % 31KB .9MB/s :
zookeeper-3.4..jar % 1480KB .5MB/s :
zookeeper-3.4..jar.md5 % .6KB/s :
zookeeper-3.4..jar.sha1 % .7KB/s :
zookeeper-3.4..jar.asc % .2MB/s :
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4.14/ node102.yinzhengjie.org.cn:/yinzhengjie/softwares/
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4./ node103.yinzhengjie.org.cn:/yinzhengjie/softwares/ ......
snapshot. % 55KB .8MB/s :
snapshot. % 140KB .7MB/s :
snapshot.83f % .4MB/s :
minikdc-krb5.conf % .0MB/s :
minikdc.ldiff % .4MB/s :
log. % 2401KB .2MB/s :
log.100001bf0 % 1001KB .5MB/s :
snapshot. % .2KB/s :
snapshot.100001bec % 2507KB .4MB/s :
check_compatibility.py % .4MB/s :
checkstyle-noframes-sorted.xsl % .4MB/s :
checkstyle.xml % .0MB/s :
findbugsExcludeFile.xml % .2MB/s :
log4j.properties % .5MB/s :
test-patch.properties % .7KB/s :
test-github-pr.sh % 24KB .3MB/s :
test-patch.sh % 24KB .5MB/s :
pom.xml % 10KB .5MB/s :
LICENSE.txt % 12KB .9MB/s :
NOTICE.txt % .8MB/s :
README.md % .9MB/s :
README_packaging.txt % .6MB/s :
build.xml % 95KB .3MB/s :
ivy.xml % 10KB .1MB/s :
ivysettings.xml % .3MB/s :
pom.xml % 31KB .6MB/s :
zookeeper-3.4..jar % 1480KB .5MB/s :
zookeeper-3.4..jar.md5 % .8KB/s :
zookeeper-3.4..jar.sha1 % .3KB/s :
zookeeper-3.4..jar.asc % .2KB/s :
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4.14/ node103.yinzhengjie.org.cn:/yinzhengjie/softwares/
7>.启动zookeeper集群
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps'
node101.yinzhengjie.org.cn | SUCCESS | rc= >>
Jps node103.yinzhengjie.org.cn | SUCCESS | rc= >>
Jps node102.yinzhengjie.org.cn | SUCCESS | rc= >>
Jps [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps'
[root@node101.yinzhengjie.org.cn ~]# xzk.sh start
启动服务
========== node101.yinzhengjie.org.cn zkServer.sh start ================
ZooKeeper JMX enabled by default
Using config: /yinzhengjie/softwares/zookeeper-3.4./bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
========== node102.yinzhengjie.org.cn zkServer.sh start ================
ZooKeeper JMX enabled by default
Using config: /yinzhengjie/softwares/zookeeper-3.4./bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
========== node103.yinzhengjie.org.cn zkServer.sh start ================
ZooKeeper JMX enabled by default
Using config: /yinzhengjie/softwares/zookeeper-3.4./bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh start
[root@node101.yinzhengjie.org.cn ~]# xzk.sh status
查看状态
========== node101.yinzhengjie.org.cn zkServer.sh status ================
ZooKeeper JMX enabled by default
Using config: /yinzhengjie/softwares/zookeeper-3.4./bin/../conf/zoo.cfg
Mode: follower
========== node102.yinzhengjie.org.cn zkServer.sh status ================
ZooKeeper JMX enabled by default
Using config: /yinzhengjie/softwares/zookeeper-3.4./bin/../conf/zoo.cfg
Mode: leader
========== node103.yinzhengjie.org.cn zkServer.sh status ================
ZooKeeper JMX enabled by default
Using config: /yinzhengjie/softwares/zookeeper-3.4./bin/../conf/zoo.cfg
Mode: follower
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh status
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps'
node103.yinzhengjie.org.cn | SUCCESS | rc= >>
QuorumPeerMain
Jps node102.yinzhengjie.org.cn | SUCCESS | rc= >>
QuorumPeerMain
Jps node101.yinzhengjie.org.cn | SUCCESS | rc= >>
QuorumPeerMain
Jps [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps'
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh
Connecting to localhost:
-- ::, [myid:] - INFO [main:Environment@] - Client environment:zookeeper.version=3.4.-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on // : GMT
-- ::, [myid:] - INFO [main:Environment@] - Client environment:host.name=node101.yinzhengjie.org.cn
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.version=1.8.0_201
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.vendor=Oracle Corporation
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.home=/yinzhengjie/softwares/jdk1..0_201/jre
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4./bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4./bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4./bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf:
2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64
2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root
2019-04-12 09:23:04,277 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89
Welcome to ZooKeeper!
2019-04-12 09:23:04,303 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2019-04-12 09:23:04,365 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2019-04-12 09:23:04,395 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x6500007f14d00000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 1] ls /zookeeper
[quota]
[zk: localhost:2181(CONNECTED) 2] ls /zookeeper/quota
[]
[zk: localhost:2181(CONNECTED) 3] quit
Quitting...
2019-04-12 09:23:13,101 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x6500007f14d00000 closed
2019-04-12 09:23:13,102 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x6500007f14d00000
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh #测试zookeeper是否可用
三.部署hadoop 3.1.2 HA案例实操（切记，3个namenode之间应该相互配置免密登录哟！）
1>.备份原有的hadoop完全分布式的配置目录
[root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/hadoop-3.1./etc/
total
drwxr-xr-x Apr : hadoop
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cp -r /yinzhengjie/softwares/hadoop-3.1./etc/hadoop/ /yinzhengjie/softwares/hadoop-3.1./etc/hadoop-full
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/hadoop-3.1./etc/
total
drwxr-xr-x Apr : hadoop
drwxr-xr-x root root Apr : hadoop-full
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cp -r /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/ /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop-full
2>.编辑hadoop的环境变量配置文件，主要修改各个服务的所属用户
[root@node101.yinzhengjie.org.cn ~]# vi /yinzhengjie/softwares/hadoop-3.1./etc/hadoop/hadoop-env.sh
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# tail - /yinzhengjie/softwares/hadoop-3.1./etc/hadoop/hadoop-env.sh
#Add Hadoop Path by yinzhengjie
export JAVA_HOME=/yinzhengjie/softwares/jdk
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_ZKFC_USER=root
export HDFS_JOURNALNODE_USER=root
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# vi /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/hadoop-env.sh
3>.编辑core-site.xml配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1./etc/hadoop/core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://yinzhengjie-hdfs</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/data/hadoop/ha</value>
</property>
<property>
<name>hadoop.http.staticuser.user</name>
<value>root</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>node101.yinzhengjie.org.cn:,node102.yinzhengjie.org.cn:,node103.yinzhengjie.org.cn:</value>
</property>
</configuration> <!-- core-site.xml配置文件的作用：
用于定义系统级别的参数，如HDFS URL、Hadoop的临时目录以及用于rack-aware集群中的配置文件的配置等，此中的参数定义会覆盖core-default.xml文件中的默认配置。 fs.defaultFS 参数的作用：
#声明namenode的地址，相当于声明hdfs文件系统。我们可以指定某个ip地址，在ha模式中我们通常指定hdfs集群的逻辑名称 hadoop.tmp.dir 参数的作用：
#声明hadoop工作目录的地址。 hadoop.http.staticuser.user 参数的作用：
#在网页界面访问数据使用的用户名。默认值是一个不真实存在的用户，此用户权限很小，不能访问不同用户的数据。这保证了数据安全。也可以设置为hdfs和hadoop等具有较高权限的用户，但会导致能够登陆网页界面的人能看到其它用户数据。实际设置请综合考虑。如无特殊需求。使用默认值就好. ha.zookeeper.quorum 参数的作用：
#指定zookeeper集群的地址，让zookeeper帮我们管理hdfs集群。千万别手动在后面加zookeeper的根目录哟，否则在使用hdfs进行zookeeper格式化会报错！ -->
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/core-site.xml
4>.编辑hdfs-site.xml配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1./etc/hadoop/hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration>
<property>
<name>dfs.replication</name>
<value></value>
</property>
<property>
<name>dfs.nameservices</name>
<value>yinzhengjie-hdfs</value>
</property>
<property>
<name>dfs.ha.namenodes.yinzhengjie-hdfs</name>
<value>namenode1,namenode2,namenode3</value>
</property>
<property>
<name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1</name>
<value>node101.yinzhengjie.org.cn:</value>
</property>
<property>
<name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode2</name>
<value>node102.yinzhengjie.org.cn:</value>
</property>
<property>
<name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode3</name>
<value>node103.yinzhengjie.org.cn:</value>
</property>
<property>
<name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode1</name>
<value>node101.yinzhengjie.org.cn:</value>
</property>
<property>
<name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode2</name>
<value>node102.yinzhengjie.org.cn:</value>
</property>
<property>
<name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode3</name>
<value>node103.yinzhengjie.org.cn:</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://node101.yinzhengjie.org.cn:8485;node102.yinzhengjie.org.cn:8485;node103.yinzhengjie.org.cn:8485/yinzhengjie-hdfs</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.yinzhengjie-hdfs</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_rsa</value>
</property>
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/data/hadoop/ha/journalnode</value>
</property>
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
</configuration> <!--
hdfs-site.xml 配置文件的作用：
#HDFS的相关设定，如文件副本的个数、块大小及是否使用强制权限等，此中的参数定义会覆盖hdfs-default.xml文件中的默认配置. dfs.replication 参数的作用：
#为了数据可用性及冗余的目的，HDFS会在多个节点上保存同一个数据块的多个副本，其默认为3个。而只有一个节点的伪分布式环境中其仅用
保存一个副本即可，这可以通过dfs.replication属性进行定义。它是一个软件级备份。 dfs.nameservices 参数的作用：
#指定一个逻辑名称，该名称会自动去帮我们找打真实NameNode节点，这和咱们的keepalive的VIP有点类似！这个名称咱们可以自定义！但是我们正在引用时千万别引用错了！ dfs.ha.namenodes.yinzhengjie-hdfs 参数的作用：
#看到没，上面我们定义了dfs.nameservices的名称，现在我们就用到了，我们把定义的逻辑名称指向了各个namenode的别名 dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1 参数的作用：
#定义远程主机调用的服务器地址。 dfs.namenode.http-address.yinzhengjie-hdfs.namenode1 参数的作用：
#上面我们定义了集群的逻辑名称，也定义节点的逻辑名称，但是真正的ip地址我们始终没有指定，因此我们这一步骤就是指定dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1对应的IP地址。当然你写主机名称也是可以的，只不过嘛你得在/etc/hosts文件做映射或者配置DNS中有对应的A记录哟！ dfs.namenode.shared.edits.dir 参数的作用：
#配置JournalNode集群的地址。 dfs.client.failover.proxy.provider.yinzhengjie-hdfs 参数的作用：
#配置故障转移的代理类，这是HDFS客户端找到active NameNode类名的一个代理类，如果你不配置这个项的话，那么客户端连接active namenode地址时会报错！实际上故障转移就是通过这个代理类来实现的，我们只需要默认官网的配置即可！不需要手动修改！ dfs.ha.fencing.methods 参数的作用：
#指定ha出现故障时的隔离方法！ dfs.ha.fencing.ssh.private-key-files 参数的作用：
#指定隔离主机的私钥PATH。 dfs.journalnode.edits.dir 参数的作用：
#指定JN节点存放编辑日志的目录。 dfs.ha.automatic-failover.enabled 参数的作用：
#开启自动故障转移功能. ->
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/hdfs-site.xml
5>.查看works配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1./etc/hadoop/workers
node101.yinzhengjie.org.cn
node102.yinzhengjie.org.cn
node103.yinzhengjie.org.cn
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/workers
5>.将修改的配置文件分发到其他节点中
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cd /yinzhengjie/softwares/hadoop-3.1./etc/hadoop
[root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1./etc/hadoop]#
[root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1./etc/hadoop]# scp core-site.xml hdfs-site.xml hadoop-env.sh node102.yinzhengjie.org.cn:`pwd`
core-site.xml % .0MB/s :
hdfs-site.xml % .4MB/s :
hadoop-env.sh % 16KB .5MB/s :
[root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1./etc/hadoop]#
[root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]# scp core-site.xml hdfs-site.xml hadoop-env.sh node102.yinzhengjie.org.cn:`pwd`
[root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1./etc/hadoop]# scp core-site.xml hdfs-site.xml hadoop-env.sh node103.yinzhengjie.org.cn:`pwd`
core-site.xml % .6KB/s :
hdfs-site.xml % .2MB/s :
hadoop-env.sh % 16KB .3MB/s :
[root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1./etc/hadoop]#
[root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]# scp core-site.xml hdfs-site.xml hadoop-env.sh node103.yinzhengjie.org.cn:`pwd`
6>.启动journalnode服务
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps'
node101.yinzhengjie.org.cn | SUCCESS | rc= >>
Jps
QuorumPeerMain node103.yinzhengjie.org.cn | SUCCESS | rc= >>
Jps
QuorumPeerMain node102.yinzhengjie.org.cn | SUCCESS | rc= >>
QuorumPeerMain
Jps [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps'
[root@node101.yinzhengjie.org.cn ~]# ll /data/hadoop/
total
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /data/hadoop/
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode
[root@node101.yinzhengjie.org.cn ~]# jps
JournalNode
QuorumPeerMain
Jps
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode
[root@node102.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode
[root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# jps
QuorumPeerMain
Jps
JournalNode
[root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode
[root@node103.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# jps
QuorumPeerMain
Jps
JournalNode
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode
ERROR: Cannot set priority of journalnode process
[root@node101.yinzhengjie.org.cn ~]# 温馨提示：
如果遇到上述问题的话，请检查你的配置文件是否有误！多一个空格都不行，如果你用我的博客的配置文件的话，建议不要把注释拷贝进去，只拷贝有效的XML代码即可！
如果遇到ERROR: Cannot set priority of journalnode process 的报错解决方案！
[root@node101.yinzhengjie.org.cn ~]# ll -R /data/
/data/:
total
drwxr-xr-x root root Apr : hadoop /data/hadoop:
total
drwxr-xr-x root root Apr : ha /data/hadoop/ha:
total
drwxr-xr-x root root Apr : journalnode /data/hadoop/ha/journalnode:
total
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll -R /data/ #注意，此时并没有数据哟！启动journalnode后，会初始化JN节点的工作目录，并没有生成具体数据！
[root@node102.yinzhengjie.org.cn ~]# ll -R /data/
/data/:
total
drwxr-xr-x root root Apr : hadoop /data/hadoop:
total
drwxr-xr-x root root Apr : ha /data/hadoop/ha:
total
drwxr-xr-x root root Apr : journalnode /data/hadoop/ha/journalnode:
total
[root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# ll -R /data/
[root@node103.yinzhengjie.org.cn ~]# ll -R /data/
/data/:
total
drwxr-xr-x root root Apr : hadoop /data/hadoop:
total
drwxr-xr-x root root Apr : ha /data/hadoop/ha:
total
drwxr-xr-x root root Apr : journalnode /data/hadoop/ha/journalnode:
total
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# ll -R /data/
[root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION
#Mon Apr :: PDT
namespaceID=
clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6
cTime=
storageType=JOURNAL_NODE
layoutVersion=-
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION #对比3个节点的该文件内容，注意观察！
[root@node102.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION
#Mon Apr :: PDT
namespaceID=
clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6
cTime=
storageType=JOURNAL_NODE
layoutVersion=-
[root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION
[root@node103.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION
#Mon Apr :: PDT
namespaceID=
clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6
cTime=
storageType=JOURNAL_NODE
layoutVersion=-
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION
7>.格式化HDFS文件系统（目的是生成初始的元数据信息，生成的信息会存放在我们hdfs-core-site.xml配置文件中指定的工作目录下，注意生成的信息和上面的JN的VERSION之间的关系！）
[root@node101.yinzhengjie.org.cn ~]# hdfs namenode -format
-- ::, INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = node101.yinzhengjie.org.cn/172.30.1.101
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 3.1.2
STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar
STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a; compiled by 'sunilg' on 2019-01-29T01:39Z
STARTUP_MSG: java = 1.8.0_201
************************************************************/
-- ::, INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
-- ::, INFO namenode.NameNode: createNameNode [-format]
Formatting using clusterid: CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6
-- ::, INFO namenode.FSEditLog: Edit logging is async:true
-- ::, INFO namenode.FSNamesystem: KeyProvider: null
-- ::, INFO namenode.FSNamesystem: fsLock is fair: true
-- ::, INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
-- ::, INFO namenode.FSNamesystem: fsOwner = root (auth:SIMPLE)
-- ::, INFO namenode.FSNamesystem: supergroup = supergroup
-- ::, INFO namenode.FSNamesystem: isPermissionEnabled = true
-- ::, INFO namenode.FSNamesystem: Determined nameservice ID: yinzhengjie-hdfs
-- ::, INFO namenode.FSNamesystem: HA Enabled: true
-- ::, INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to . Disabling file IO profiling
-- ::, INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=, counted=, effected=
-- ::, INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
-- ::, INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to :::00.000
-- ::, INFO blockmanagement.BlockManager: The block deletion will start around Apr ::
-- ::, INFO util.GSet: Computing capacity for map BlocksMap
-- ::, INFO util.GSet: VM type = -bit
-- ::, INFO util.GSet: 2.0% max memory 839.5 MB = 16.8 MB
-- ::, INFO util.GSet: capacity = ^ = entries
-- ::, INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
-- ::, INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension() assuming MILLISECONDS
-- ::, INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
-- ::, INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes =
-- ::, INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension =
-- ::, INFO blockmanagement.BlockManager: defaultReplication =
-- ::, INFO blockmanagement.BlockManager: maxReplication =
-- ::, INFO blockmanagement.BlockManager: minReplication =
-- ::, INFO blockmanagement.BlockManager: maxReplicationStreams =
-- ::, INFO blockmanagement.BlockManager: redundancyRecheckInterval = 3000ms
-- ::, INFO blockmanagement.BlockManager: encryptDataTransfer = false
-- ::, INFO blockmanagement.BlockManager: maxNumBlocksToLog =
-- ::, INFO namenode.FSDirectory: GLOBAL serial map: bits= maxEntries=
-- ::, INFO util.GSet: Computing capacity for map INodeMap
-- ::, INFO util.GSet: VM type = -bit
-- ::, INFO util.GSet: 1.0% max memory 839.5 MB = 8.4 MB
-- ::, INFO util.GSet: capacity = ^ = entries
-- ::, INFO namenode.FSDirectory: ACLs enabled? false
-- ::, INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
-- ::, INFO namenode.FSDirectory: XAttrs enabled? true
-- ::, INFO namenode.NameNode: Caching file names occurring more than times
-- ::, INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit:
-- ::, INFO snapshot.SnapshotManager: SkipList is disabled
-- ::, INFO util.GSet: Computing capacity for map cachedBlocks
-- ::, INFO util.GSet: VM type = -bit
-- ::, INFO util.GSet: 0.25% max memory 839.5 MB = 2.1 MB
-- ::, INFO util.GSet: capacity = ^ = entries
-- ::, INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets =
-- ::, INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users =
-- ::, INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = ,,
-- ::, INFO namenode.FSNamesystem: Retry cache on namenode is enabled
-- ::, INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is millis
-- ::, INFO util.GSet: Computing capacity for map NameNodeRetryCache
-- ::, INFO util.GSet: VM type = -bit
-- ::, INFO util.GSet: 0.029999999329447746% max memory 839.5 MB = 257.9 KB
-- ::, INFO util.GSet: capacity = ^ = entries
-- ::, INFO namenode.FSImage: Allocated new BlockPoolId: BP--172.30.1.101-
-- ::, INFO common.Storage: Storage directory /data/hadoop/ha/dfs/name has been successfully formatted.
-- ::, INFO namenode.FSImageFormatProtobuf: Saving image file /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
-- ::, INFO namenode.FSImageFormatProtobuf: Image file /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 of size bytes saved in seconds .
-- ::, INFO namenode.NNStorageRetentionManager: Going to retain images with txid >=
-- ::, INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at node101.yinzhengjie.org.cn/172.30.1.101
************************************************************/
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs namenode -format
[root@node101.yinzhengjie.org.cn ~]# ll -R /data/hadoop/ha/dfs
/data/hadoop/ha/dfs:
total
drwxr-xr-x root root Apr : name /data/hadoop/ha/dfs/name:
total
drwxr-xr-x root root Apr : current /data/hadoop/ha/dfs/name/current:
total
-rw-r--r-- root root Apr : fsimage_0000000000000000000
-rw-r--r-- root root Apr : fsimage_0000000000000000000.md5
-rw-r--r-- root root Apr : seen_txid
-rw-r--r-- root root Apr : VERSION
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION
#Mon Apr :: PDT
namespaceID=
clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6
cTime=
storageType=NAME_NODE
blockpoolID=BP--172.30.1.101-
layoutVersion=-
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll -R /data/hadoop/ha/dfs
[root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION
#Mon Apr :: PDT
namespaceID=
clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6
cTime=
storageType=NAME_NODE
blockpoolID=BP--172.30.1.101-
layoutVersion=-
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION #大家注意，对比里面的每一个参数和上面的JN节点的VERSION进行对比哟！
[root@node102.yinzhengjie.org.cn ~]# ll /data/hadoop/ha/
total
drwxr-xr-x root root Apr : journalnode
[root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# ll /data/hadoop/ha/
[root@node103.yinzhengjie.org.cn ~]# ll /data/hadoop/ha/
total
drwxr-xr-x root root Apr : journalnode
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# ll /data/hadoop/ha/
8>.启动NameNode并同步数据
[root@node101.yinzhengjie.org.cn ~]# jps
Jps
JournalNode
QuorumPeerMain
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start namenode
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# jps
JournalNode
NameNode
Jps
QuorumPeerMain
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start namenode #启动NameNode服务
[root@node103.yinzhengjie.org.cn ~]# hdfs namenode -bootstrapStandby
-- ::, INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = node103.yinzhengjie.org.cn/172.30.1.103
STARTUP_MSG: args = [-bootstrapStandby]
STARTUP_MSG: version = 3.1.2
STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar
STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a; compiled by 'sunilg' on 2019-01-29T01:39Z
STARTUP_MSG: java = 1.8.0_201
************************************************************/
-- ::, INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
-- ::, INFO namenode.NameNode: createNameNode [-bootstrapStandby]
-- ::, INFO ha.BootstrapStandby: Found nn: namenode1, ipc: node101.yinzhengjie.org.cn/172.30.1.101:
-- ::, INFO ha.BootstrapStandby: Found nn: namenode2, ipc: node102.yinzhengjie.org.cn/172.30.1.102:
=====================================================
About to bootstrap Standby ID namenode3 from:
Nameservice ID: yinzhengjie-hdfs
Other Namenode ID: namenode1
Other NN's HTTP address: http://node101.yinzhengjie.org.cn:9870
Other NN's IPC address: node101.yinzhengjie.org.cn/172.30.1.101:8020
Namespace ID:
Block pool ID: BP--172.30.1.101-
Cluster ID: CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6
Layout version: -
isUpgradeFinalized: true
=====================================================
-- ::, INFO common.Storage: Storage directory /data/hadoop/ha/dfs/name has been successfully formatted.
-- ::, INFO namenode.FSEditLog: Edit logging is async:true
-- ::, INFO namenode.TransferFsImage: Opening connection to http://node101.yinzhengjie.org.cn:9870/imagetransfer?getimage=1&txid=0&storageInfo=-64:1130551675:1555340006352:CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6&bootstrapstandby=true
-- ::, INFO common.Util: Combined time for file download and fsync to all disks took .01s. The file download took .01s at 0.00 KB/s. Synchronous (fsync) write to disk of /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 took .00s.
-- ::, INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000000 size bytes.
-- ::, INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at node103.yinzhengjie.org.cn/172.30.1.103
************************************************************/
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# hdfs namenode -bootstrapStandby #注意，一定要上面的NameNode启动成功后才能执行这条命令进行同步数据哟！
[root@node103.yinzhengjie.org.cn ~]# ll -R /data/hadoop/ha/dfs
/data/hadoop/ha/dfs:
total
drwxr-xr-x root root Apr : name /data/hadoop/ha/dfs/name:
total
drwxr-xr-x root root Apr : current /data/hadoop/ha/dfs/name/current:
total
-rw-r--r-- root root Apr : fsimage_0000000000000000000
-rw-r--r-- root root Apr : fsimage_0000000000000000000.md5
-rw-r--r-- root root Apr : seen_txid
-rw-r--r-- root root Apr : VERSION
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# ll -R /data/hadoop/ha/dfs #这个目录在上一步是没有的，由于我们同步成功了，因此我们可以感到该目录下的所有数据！
[root@node103.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION
#Mon Apr :: PDT
namespaceID=
clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6
cTime=
storageType=NAME_NODE
blockpoolID=BP--172.30.1.101-
layoutVersion=-
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION
四.zookeeper格式化、启动ha及测试
1>.初始化hdfs在zookeeper注册节点目录
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh
Connecting to localhost:
-- ::, [myid:] - INFO [main:Environment@] - Client environment:zookeeper.version=3.4.-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on // : GMT
-- ::, [myid:] - INFO [main:Environment@] - Client environment:host.name=node101.yinzhengjie.org.cn
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.version=1.8.0_201
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.vendor=Oracle Corporation
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.home=/yinzhengjie/softwares/jdk1..0_201/jre
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4./bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4./bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4./bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf:
2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64
2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root
2019-04-15 08:15:10,374 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89
Welcome to ZooKeeper!
2019-04-15 08:15:10,406 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2019-04-15 08:15:10,476 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2019-04-15 08:15:10,497 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x65000f71aa200000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 1]
[zk: localhost:2181(CONNECTED) 1] quit
Quitting...
2019-04-15 08:24:18,604 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x65000f71aa200001 closed
2019-04-15 08:24:18,606 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x65000f71aa200001
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh
[root@node101.yinzhengjie.org.cn ~]# hdfs zkfc -formatZK
-- ::, INFO tools.DFSZKFailoverController: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting DFSZKFailoverController
STARTUP_MSG: host = node101.yinzhengjie.org.cn/172.30.1.101
STARTUP_MSG: args = [-formatZK]
STARTUP_MSG: version = 3.1.2
STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar
STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a; compiled by 'sunilg' on 2019-01-29T01:39Z
STARTUP_MSG: java = 1.8.0_201
************************************************************/
-- ::, INFO tools.DFSZKFailoverController: registered UNIX signal handlers for [TERM, HUP, INT]
-- ::, INFO tools.DFSZKFailoverController: Failover controller configured for NameNode NameNode at node101.yinzhengjie.org.cn/172.30.1.101:
-- ::, INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on // : GMT
-- ::, INFO zookeeper.ZooKeeper: Client environment:host.name=node101.yinzhengjie.org.cn
-- ::, INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_201
-- ::, INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
-- ::, INFO zookeeper.ZooKeeper: Client environment:java.home=/yinzhengjie/softwares/jdk1..0_201/jre
-- ::, INFO zookeeper.ZooKeeper: Client environment:java.class.path=/yinzhengjie/softwares/hadoop-3.1./etc/hadoop:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/asm-5.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/audience-annotations-0.5..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/avro-1.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-beanutils-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-collections-3.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-configuration2-2.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-logging-1.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-math3-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/curator-client-2.13..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/curator-framework-2.13..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/curator-recipes-2.13..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/gson-2.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/guava-11.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/hadoop-annotations-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/hadoop-auth-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/htrace-core4-4.1.-incubating.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/httpclient-4.5..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/httpcore-4.4..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jackson-annotations-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jackson-core-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jackson-core-asl-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jackson-databind-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jackson-jaxrs-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jackson-mapper-asl-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jackson-xc-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/javax.servlet-api-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jaxb-api-2.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jaxb-impl-2.2.-.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jcip-annotations-1.0-.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jetty-http-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jetty-io-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jetty-security-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jetty-server-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jetty-servlet-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jetty-util-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jetty-webapp-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jetty-xml-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jsch-0.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jsr305-3.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jsr311-api-1.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerb-admin-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerb-client-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerb-common-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerb-core-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerb-crypto-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerb-identity-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerb-server-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerb-simplekdc-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerb-util-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerby-asn1-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerby-config-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerby-pkix-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerby-util-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/kerby-xdr-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/log4j-1.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/netty-3.10..Final.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/nimbus-jose-jwt-4.41..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/protobuf-java-2.5..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/slf4j-api-1.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/slf4j-log4j12-1.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/snappy-java-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/stax2-api-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/token-provider-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/woodstox-core-5.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/zookeeper-3.4..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/jul-to-slf4j-1.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/lib/metrics-core-3.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/hadoop-common-3.1.-tests.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/hadoop-common-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/hadoop-nfs-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/common/hadoop-kms-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-daemon-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jetty-util-ajax-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/netty-all-4.0..Final.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/okhttp-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/okio-1.6..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/hadoop-auth-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/log4j-1.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/httpclient-4.5..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/httpcore-4.4..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-logging-1.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jcip-annotations-1.0-.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/asm-5.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/zookeeper-3.4..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/audience-annotations-0.5..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/netty-3.10..Final.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/curator-framework-2.13..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/curator-client-2.13..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/guava-11.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jsr305-3.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerb-simplekdc-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerb-client-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerby-config-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerb-core-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerby-pkix-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerby-asn1-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerby-util-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerb-common-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerb-crypto-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerb-util-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/token-provider-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerb-admin-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerb-server-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerb-identity-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/kerby-xdr-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jsr311-api-1.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/javax.servlet-api-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/json-simple-1.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jetty-server-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jetty-http-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jetty-util-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jetty-io-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jetty-webapp-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jetty-xml-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jetty-servlet-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jetty-security-9.3..v20180605.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/hadoop-annotations-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-math3-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-collections-3.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jaxb-impl-2.2.-.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jaxb-api-2.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jackson-core-asl-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jackson-mapper-asl-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jackson-jaxrs-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jackson-xc-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-beanutils-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-configuration2-2.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/avro-1.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/snappy-java-1.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/protobuf-java-2.5..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/gson-2.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jsch-0.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/curator-recipes-2.13..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/htrace-core4-4.1.-incubating.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jackson-databind-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jackson-annotations-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/jackson-core-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/stax2-api-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/lib/woodstox-core-5.0..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-3.1.-tests.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-nfs-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-client-3.1.-tests.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-client-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.-tests.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-native-client-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.-tests.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-rbf-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.-tests.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/HikariCP-java7-2.4..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/dnsjava-2.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/ehcache-3.3..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/jackson-jaxrs-base-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/java-util-1.9..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/javax.inject-.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/json-io-2.5..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/metrics-core-3.2..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/mssql-jdbc-6.2..jre7.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/lib/swagger-annotations-1.5..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-api-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-client-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-common-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-registry-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-server-common-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-server-router-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-server-tests-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-services-api-3.1..jar:/yinzhengjie/softwares/hadoop-3.1./share/hadoop/yarn/hadoop-yarn-services-core-3.1..jar
-- ::, INFO zookeeper.ZooKeeper: Client environment:java.library.path=/yinzhengjie/softwares/hadoop-3.1./lib/native
-- ::, INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
-- ::, INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
-- ::, INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
-- ::, INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
-- ::, INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.-.el7.x86_64
-- ::, INFO zookeeper.ZooKeeper: Client environment:user.name=root
-- ::, INFO zookeeper.ZooKeeper: Client environment:user.home=/root
-- ::, INFO zookeeper.ZooKeeper: Client environment:user.dir=/root
-- ::, INFO zookeeper.ZooKeeper: Initiating client connection, connectString=node101.yinzhengjie.org.cn:,node102.yinzhengjie.org.cn:,node103.yinzhengjie.org.cn: sessionTimeout= watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@3c407114
-- ::, INFO zookeeper.ClientCnxn: Opening socket connection to server node103.yinzhengjie.org.cn/172.30.1.103:. Will not attempt to authenticate using SASL (unknown error)
-- ::, INFO zookeeper.ClientCnxn: Socket connection established to node103.yinzhengjie.org.cn/172.30.1.103:, initiating session
-- ::, INFO zookeeper.ClientCnxn: Session establishment complete on server node103.yinzhengjie.org.cn/172.30.1.103:, sessionid = 0x67000f716ed80001, negotiated timeout =
-- ::, INFO ha.ActiveStandbyElector: Session connected.
-- ::, INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/yinzhengjie-hdfs in ZK.
-- ::, INFO zookeeper.ZooKeeper: Session: 0x67000f716ed80001 closed
-- ::, INFO zookeeper.ClientCnxn: EventThread shut down for session: 0x67000f716ed80001
-- ::, INFO tools.DFSZKFailoverController: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DFSZKFailoverController at node101.yinzhengjie.org.cn/172.30.1.101
************************************************************/
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs zkfc -formatZK #紧张zookeeper格式化操作
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh
Connecting to localhost:
-- ::, [myid:] - INFO [main:Environment@] - Client environment:zookeeper.version=3.4.-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on // : GMT
-- ::, [myid:] - INFO [main:Environment@] - Client environment:host.name=node101.yinzhengjie.org.cn
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.version=1.8.0_201
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.vendor=Oracle Corporation
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.home=/yinzhengjie/softwares/jdk1..0_201/jre
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4./bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4./bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4./bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf:
2019-04-15 08:24:07,204 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64
2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2019-04-15 08:24:07,206 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root
2019-04-15 08:24:07,207 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89
Welcome to ZooKeeper!
2019-04-15 08:24:07,230 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2019-04-15 08:24:07,303 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2019-04-15 08:24:07,322 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x65000f71aa200001, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper, hadoop-ha]
[zk: localhost:2181(CONNECTED) 1] ls /hadoop-ha
[yinzhengjie-hdfs]
[zk: localhost:2181(CONNECTED) 2]
[zk: localhost:2181(CONNECTED) 2] ls /hadoop-ha/yinzhengjie-hdfs
[]
[zk: localhost:2181(CONNECTED) 3]
[zk: localhost:2181(CONNECTED) 3] quit
Quitting...
2019-04-15 08:24:18,604 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x65000f71aa200001 closed
2019-04-15 08:24:18,606 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x65000f71aa200001
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh #登录zookeeper可以查看有新的数据生成了
2>.启动hadoop集群
[root@node101.yinzhengjie.org.cn ~]# start-dfs.sh
Starting namenodes on [node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn]
Last login: Mon Apr :: PDT from 172.30.1.1 on pts/
node101.yinzhengjie.org.cn: namenode is running as process . Stop it first.
Starting datanodes
Last login: Mon Apr :: PDT on pts/
Starting journal nodes [node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn]
Last login: Mon Apr :: PDT on pts/
node101.yinzhengjie.org.cn: journalnode is running as process . Stop it first.
node102.yinzhengjie.org.cn: journalnode is running as process . Stop it first.
node103.yinzhengjie.org.cn: journalnode is running as process . Stop it first.
Starting ZK Failover Controllers on NN hosts [node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn]
Last login: Mon Apr :: PDT on pts/
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# start-dfs.sh
[root@node101.yinzhengjie.org.cn ~]# jps
DataNode
JournalNode
DFSZKFailoverController
NameNode
Jps
QuorumPeerMain
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# jps
[root@node102.yinzhengjie.org.cn ~]# jps
QuorumPeerMain
DataNode
JournalNode
Jps
[root@node102.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# jps
[root@node103.yinzhengjie.org.cn ~]# jps
QuorumPeerMain
Jps
JournalNode
DataNode
NameNode
[root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# jps
3>.启动hadoop集群成功后，我们再次查看zookeeper集群管理hadoop的节点信息
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh
Connecting to localhost:
-- ::, [myid:] - INFO [main:Environment@] - Client environment:zookeeper.version=3.4.-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on // : GMT
-- ::, [myid:] - INFO [main:Environment@] - Client environment:host.name=node101.yinzhengjie.org.cn
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.version=1.8.0_201
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.vendor=Oracle Corporation
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.home=/yinzhengjie/softwares/jdk1..0_201/jre
-- ::, [myid:] - INFO [main:Environment@] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4./bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4./bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4./bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf:
2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64
2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2019-04-15 08:31:09,363 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2019-04-15 08:31:09,363 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root
2019-04-15 08:31:09,365 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89
Welcome to ZooKeeper!
2019-04-15 08:31:09,399 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2019-04-15 08:31:09,507 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2019-04-15 08:31:09,531 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x65000f71aa200002, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /hadoop-ha/yinzhengjie-hdfs
[ActiveBreadCrumb, ActiveStandbyElectorLock]
[zk: localhost:2181(CONNECTED) 1]
[zk: localhost:2181(CONNECTED) 1] ls /hadoop-ha/yinzhengjie-hdfs/ActiveBreadCrumb
[]
[zk: localhost:2181(CONNECTED) 2]
[zk: localhost:2181(CONNECTED) 2] ls /hadoop-ha/yinzhengjie-hdfs/ActiveStandbyElectorLock
[]
[zk: localhost:2181(CONNECTED) 3]
[zk: localhost:2181(CONNECTED) 3] get /hadoop-ha/yinzhengjie-hdfs/ActiveBreadCrumb yinzhengjie-hdfs namenode1node101.yinzhengjie.org.cn �>(�> #我们这里可以看到注册节点成功的为node101.yinzhengjie.org.cn
cZxid = 0x800000011
ctime = Mon Apr 15 08:28:56 PDT 2019
mZxid = 0x800000011
mtime = Mon Apr 15 08:28:56 PDT 2019
pZxid = 0x800000011
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 63
numChildren = 0
[zk: localhost:2181(CONNECTED) 4]
[zk: localhost:2181(CONNECTED) 4] get /hadoop-ha/yinzhengjie-hdfs/ActiveStandbyElectorLock yinzhengjie-hdfs namenode1node101.yinzhengjie.org.cn �>(�> #注意，这里注册节点成功的为node101.yinzhengjie.org.cn
cZxid = 0x800000010
ctime = Mon Apr 15 08:28:56 PDT 2019
mZxid = 0x800000010
mtime = Mon Apr 15 08:28:56 PDT 2019
pZxid = 0x800000010
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x66000f71a2960001
dataLength = 63
numChildren = 0
[zk: localhost:2181(CONNECTED) 5]
4>.查看node101.yinzhengjie.org.cn节点的Web UI信息
5>.查看node103.yinzhengjie.org.cn节点的Web UI信息
6>.停用node101.yinzhengjie.org.cn的namenode进程停用，观察现象（注意检查多个namenode之间是否配置了免密码登录）
[root@node101.yinzhengjie.org.cn ~]# jps
DataNode
JournalNode
DFSZKFailoverController
NameNode
ZooKeeperMain
Jps
QuorumPeerMain
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon stop namenode
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# jps
Jps
DataNode
JournalNode
DFSZKFailoverController
ZooKeeperMain
QuorumPeerMain
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon stop namenode #方式一停掉NN节点观察现象
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon stop zkfc
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# jps
ZooKeeperMain
DataNode
Jps
JournalNode
QuorumPeerMain
[root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon stop zkfc #方式二停掉ZKFC进程观察现象
大数据技术之Hadoop3.1.2版本HA模式的更多相关文章
- 大数据技术之Hadoop3.1.2版本完全分布式部署
大数据技术之Hadoop3.1.2版本完全分布式部署 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.主机环境准备 1>.操作系统环境 [root@node101.yinz ...
- 大数据技术之Hadoop3.1.2版本伪分布式部署
大数据技术之Hadoop3.1.2版本伪分布式部署 作者:尹正杰 版权声明:原创作品,谢绝转载!否则将追究法律责任. 一.主机环境准备 1>.操作系统环境 [root@node101.yinzh ...
- 开源大数据技术专场（上午）:Spark、HBase、JStorm应用与实践
16日上午9点,2016云栖大会“开源大数据技术专场” (全天)在阿里云技术专家封神的主持下开启.通过封神了解到,在上午的专场中,阿里云高级技术专家无谓.阿里云技术专家封神.阿里巴巴中间件技术部高级技 ...
- 【学习笔记】大数据技术原理与应用（MOOC视频、厦门大学林子雨）
1 大数据概述 大数据特性:4v volume velocity variety value 即大量化.快速化.多样化.价值密度低 数据量大:大数据摩尔定律 快速化:从数据的生成到消耗,时间窗口小,可 ...
- 大数据技术 - 为什么是SQL
在大数据处理以及分析中 SQL 的普及率非常高,几乎是每一个大数据工程师必须掌握的语言,甚至非数据处理岗位的人也在学习使用 SQL.今天这篇文章就聊聊 SQL 在数据分析中作用以及掌握 SQL 的必要 ...
- 大数据技术之kettle
大数据技术之kettle 第1章 kettle概述 1.1 什么是kettle kettle是一款开源的ETL工具,纯java编写,可以在Windows.Linux.Uni ...
- MaxCompute 最新特性介绍 | 2019大数据技术公开课第三季
摘要:距离上一次MaxCompute新功能的线上发布已经过去了大约一个季度的时间,而在这一段时间里,MaxCompute不断地在增加新的功能和特性,比如参数化视图.UDF支持动态参数.支持分区裁剪.生 ...
- 大数据技术之HBase
第1章 HBase简介 1.1 什么是HBase HBase的原型是Google的BigTable论文,受到了该论文思想的启发,目前作为Hadoop的子项目来开发维护,用于支持结构化的数据存储. 官方 ...
- 大数据技术之Hadoop入门
第1章 大数据概论 1.1 大数据概念 大数据概念如图2-1 所示. 图2-1 大数据概念 1.2 大数据特点(4V) 大数据特点如图2-2,2-3,2-4,2-5所示 图2-2 大数据特点之大量 ...
随机推荐
- [LeetCode] 752. Open the Lock 开锁
You have a lock in front of you with 4 circular wheels. Each wheel has 10 slots: '0', '1', '2', '3', ...
- Kubernetes 控制器之 Service 讲解(七)
一.背景介绍 我们这里准备三台机器,一台master,两台node,采用kubeadm的方式进行安装的,安装过程大家可以参照我之前的博文. IP 角色 版本 192.168.1.200 master ...
- mongoDB杂项
密码更改:db.changeUserPassword('tank2','test'); 在这里我遇到了一个问题: mongod: error while loading shared librarie ...
- Windows 7安装Service Pack 1失败问题
问题 很多朋友在电脑上安装某些软件时,会发现安装不了,提示信息大多是"本软件只支持 Windows 7 SP1 及更新版本"等等.这说明,你的电脑仍然在运行最早的Windows7版 ...
- TestNG执行顺序控制
1.class执行顺序控制---testng.xml之preserve-order preserve-order:用来控制<test>里面所有<classes>的执行顺序.&l ...
- java8新特性五-Stream
继上次学习过Java8中的非常重要的Lambda表达式之后,接下来就要学习另一个也比较重要的知识啦,也就如标题所示:Stream,而它的学习是完全依赖于之前学习的Lambda表达式. Java 8 A ...
- spring 循环引用问题，在一次问题调试过程中发现有个小伙伴竟然把循环引用设置成false了。估计是百度的时候没小心额外的代码吧。。。
循环引用属性操作: 1)AbstractAutowireCapableBeanFactory类中的allowCircularReferences被设置为了false. 2)代码: Annotation ...
- UDP中一个包的大小最大能多大？TCP呢？
(以上为以太网的帧格式) Preamble:前导同步码7个字节+帧开始定界符1个字节=8个字节 Destination MAC adress:目的MAC地址 Source MAC adress:源MA ...
- Django 自定义表名
class UserInfo(AbstractUser): phone = models.BigIntegerField(null=True, blank=True) create_time = mo ...
- Fedora30 install VS Code
We currently ship the stable 64-bit VS Code in a yum repository, the following script will install t ...