前言：

本文章是在 Hadoop2.7.6_07_HA高可用的基础上完成的，所以不清楚的可参见这篇文章。

1. Hadoop的federation机制

文件的元数据是放在namenode上的，只有一个Namespace（命名空间）。随着HDFS的数据越来越多，单个namenode的资源使用必然会达到上限，而且namenode的负载能力也会越来越高，限制HDFS的性能。

Federation即为“联邦”，该特性允许一个HDFS集群中存在多个NameNode同时对外提供服务，这些NameNode分管一部分目录（水平切分），彼此之间相互隔离，但共享底层的DataNode存储资源。

1.1. HDFS-federation图解

2. 主机规划

主机名称	外网IP	内网IP	操作系统	备注	安装软件	运行进程
mini01	10.0.0.111	172.16.1.111	CentOS 7.4	ssh port:22	jdk、hadoop	NameNode、DFSZKFailoverController(zkfc)
mini02	10.0.0.112	172.16.1.112	CentOS 7.4	ssh port:22	jdk、hadoop	NameNode、DFSZKFailoverController(zkfc)
mini03	10.0.0.113	172.16.1.113	CentOS 7.4	ssh port:22	jdk、hadoop、zookeeper	NameNode、ResourceManager
mini04	10.0.0.114	172.16.1.114	CentOS 7.4	ssh port:22	jdk、hadoop、zookeeper	NameNode、ResourceManager
mini05	10.0.0.115	172.16.1.115	CentOS 7.4	ssh port:22	jdk、hadoop、zookeeper	DataNode、NodeManager、JournalNode、QuorumPeerMain
mini06	10.0.0.116	172.16.1.116	CentOS 7.4	ssh port:22	jdk、hadoop、zookeeper	DataNode、NodeManager、JournalNode、QuorumPeerMain
mini07	10.0.0.117	172.16.1.117	CentOS 7.4	ssh port:22	jdk、hadoop、zookeeper	DataNode、NodeManager、JournalNode、QuorumPeerMain

注意：此规划是在HA的基础上进行的federation，即 mini01和mini02做HA，mini03和mini04做HA；之后再做federation。

	HA的配置请参见：Hadoop2.7.6_07_HA高可用

Linux添加hosts信息，保证每台都可以相互ping通

 [root@mini01 ~]# cat /etc/hosts

 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

 ::         localhost localhost.localdomain localhost6 localhost6.localdomain6

 10.0.0.111    mini01

 10.0.0.112    mini02

 10.0.0.113    mini03

 10.0.0.114    mini04

 10.0.0.115    mini05

 10.0.0.116    mini06

 10.0.0.117    mini07

Windows的hosts文件修改

 # 文件位置C:\Windows\System32\drivers\etc   在hosts中追加如下内容

 …………………………………………

 10.0.0.111    mini01

 10.0.0.112    mini02

 10.0.0.113    mini03

 10.0.0.114    mini04

 10.0.0.115    mini05

 10.0.0.116    mini06

 10.0.0.117    mini07

2.1. 注意事项

其中添加yun用户、实现yun用户免秘钥登录、Jdk【java8】、Zookeeper部署；请参见Hadoop2.7.6_07_HA高可用

3. Hadoop部署与配置修改

　　注意：每台机器的Hadoop以及配置相同

3.1. 部署

 [yun@mini01 software]$ pwd

 /app/software

 [yun@mini01 software]$ ll

 total

 -rw-r--r--  yun yun  Jun   : CentOS-.4_hadoop-2.7..tar.gz

 [yun@mini01 software]$ tar xf CentOS-.4_hadoop-2.7..tar.gz

 [yun@mini01 software]$ mv hadoop-2.7./ /app/

 [yun@mini01 software]$ cd

 [yun@mini01 ~]$ ln -s hadoop-2.7./ hadoop

 [yun@mini01 ~]$ ll

 total

 lrwxrwxrwx   yun yun    Jun   : hadoop -> hadoop-2.7./

 drwxr-xr-x   yun yun   Jun   : hadoop-2.7.

 lrwxrwxrwx   yun yun    May  : jdk -> jdk1..0_112

 drwxr-xr-x   yun yun   Sep    jdk1..0_112

3.2. 环境变量

 [root@mini01 profile.d]# pwd

 /etc/profile.d

 [root@mini01 profile.d]# vim hadoop.sh

 export HADOOP_HOME="/app/hadoop"

 export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

 [root@mini01 profile.d]# source /etc/profile  # 生效

3.3. core-site.xml

 [yun@mini01 hadoop]$ pwd

 /app/hadoop/etc/hadoop

 [yun@mini01 hadoop]$ vim core-site.xml

 <?xml version="1.0" encoding="UTF-8"?>

 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 <!--

   ………………

 -->

 <!-- Put site-specific property overrides in this file. -->

 <configuration>

   <property>

     <name>fs.defaultFS</name>

     <value>viewfs:///</value>

   </property>

   <property>

     <name>fs.viewfs.mounttable.default.link./bi</name>

     <value>hdfs://bi/</value>

   </property>

   <property>

     <name>fs.viewfs.mounttable.default.link./dt</name>

     <value>hdfs://dt/</value>

   </property>

   <!-- 指定hadoop临时目录 -->

   <property>

     <name>hadoop.tmp.dir</name>

     <value>/app/hadoop/tmp</value>

   </property>

   <!-- 指定zookeeper地址 -->

   <property>

     <name>ha.zookeeper.quorum</name>

     <value>mini03:,mini04:,mini05:,mini06:,mini07:</value>

   </property>

 </configuration>

3.4. hdfs-site.xml

3.4.1. 在mini01和mini02中

 # 在mini01、mini02、mini05、mini06、mini07 可使用如下配置

 [yun@mini01 hadoop]$ pwd

 /app/hadoop/etc/hadoop

 [yun@mini01 hadoop]$ vim hdfs-site.xml

 <?xml version="1.0" encoding="UTF-8"?>

 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 <!--

   ……………………

 -->

 <!-- Put site-specific property overrides in this file. -->

 <configuration>

   <!--指定hdfs的nameservice为bi,dt，需要和core-site.xml中的保持一致 -->

   <property>

     <name>dfs.nameservices</name>

     <value>bi,dt</value>

   </property>

   <!-- bi下面有两个NameNode，分别是nn1，nn2 -->

   <property>

     <name>dfs.ha.namenodes.bi</name>

     <value>nn1,nn2</value>

   </property>

   <!-- dt下面有两个NameNode，分别是nn3，nn4 -->

   <property>

     <name>dfs.ha.namenodes.dt</name>

     <value>nn3,nn4</value>

   </property>

   <!-- nn1的RPC通信地址 -->

   <property>

     <name>dfs.namenode.rpc-address.bi.nn1</name>

     <value>mini01:</value>

   </property>

   <!-- nn1的http通信地址 -->

   <property>

     <name>dfs.namenode.http-address.bi.nn1</name>

     <value>mini01:</value>

   </property>

   <!-- nn2的RPC通信地址 -->

   <property>

     <name>dfs.namenode.rpc-address.bi.nn2</name>

     <value>mini02:</value>

   </property>

   <!-- nn2的http通信地址 -->

   <property>

     <name>dfs.namenode.http-address.bi.nn2</name>

     <value>mini02:</value>

   </property>

   <!-- nn3的RPC通信地址 -->

   <property>

     <name>dfs.namenode.rpc-address.dt.nn3</name>

     <value>mini03:</value>

   </property>

   <!-- nn3的http通信地址 -->

   <property>

     <name>dfs.namenode.http-address.dt.nn3</name>

     <value>mini03:</value>

   </property>

   <!-- nn4的RPC通信地址 -->

   <property>

     <name>dfs.namenode.rpc-address.dt.nn4</name>

     <value>mini04:</value>

   </property>

   <!-- nn4的http通信地址 -->

   <property>

     <name>dfs.namenode.http-address.dt.nn4</name>

     <value>mini04:</value>

   </property>

   <!-- 指定NameNode的edits元数据在JournalNode上的存放位置 -->

   <!--  在bi名称空间的两个namenode中用如下配置  -->

   <property>

     <name>dfs.namenode.shared.edits.dir</name>

     <value>qjournal://mini05:8485;mini06:8485;mini07:8485/bi</value>

   </property>

   <!--  在dt名称空间的两个namenode中用如下配置  # 在mini01和mini02中注释掉  -->

   <!--

   <property>

     <name>dfs.namenode.shared.edits.dir</name>

     <value>qjournal://mini05:8485;mini06:8485;mini07:8485/dt</value>

   </property>

    -->

   <!-- 指定JournalNode在本地磁盘存放数据的位置 -->

   <property>

     <name>dfs.journalnode.edits.dir</name>

     <value>/app/hadoop/journaldata</value>

   </property>

   <!-- 开启NameNode失败自动切换 -->

   <property>

     <name>dfs.ha.automatic-failover.enabled</name>

     <value>true</value>

   </property>

   <!-- 配置失败自动切换实现方式 -->

   <property>

     <name>dfs.client.failover.proxy.provider.bi</name>

     <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>

   </property>

   <property>

     <name>dfs.client.failover.proxy.provider.dt</name>

     <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>

   </property>

   <!-- 配置隔离机制方法，多个机制用换行分割，即每个机制暂用一行-->

   <!-- 其中shell(/bin/true) 表示可执行一个脚本  比如 shell(/app/yunwei/hadoop_fence.sh) -->

   <property>

     <name>dfs.ha.fencing.methods</name>

     <value>

       sshfence

       shell(/bin/true)

     </value>

   </property>

   <!-- 使用sshfence隔离机制时需要ssh免登陆 -->

   <property>

     <name>dfs.ha.fencing.ssh.private-key-files</name>

     <value>/app/.ssh/id_rsa</value>

   </property>

   <!-- 配置sshfence隔离机制超时时间 单位：毫秒 -->

   <property>

     <name>dfs.ha.fencing.ssh.connect-timeout</name>

     <value></value>

   </property>

 </configuration>

3.4.2. 在mini03和mini04中

 # 在mini03、mini04 可使用如下配置

 [yun@mini01 hadoop]$ pwd

 /app/hadoop/etc/hadoop

 [yun@mini01 hadoop]$ vim hdfs-site.xml

 <?xml version="1.0" encoding="UTF-8"?>

 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 <!--

   ……………………

 -->

 <!-- Put site-specific property overrides in this file. -->

 <configuration>

   <!--指定hdfs的nameservice为bi,dt，需要和core-site.xml中的保持一致 -->

   <property>

     <name>dfs.nameservices</name>

     <value>bi,dt</value>

   </property>

   <!-- bi下面有两个NameNode，分别是nn1，nn2 -->

   <property>

     <name>dfs.ha.namenodes.bi</name>

     <value>nn1,nn2</value>

   </property>

   <!-- dt下面有两个NameNode，分别是nn3，nn4 -->

   <property>

     <name>dfs.ha.namenodes.dt</name>

     <value>nn3,nn4</value>

   </property>

   <!-- nn1的RPC通信地址 -->

   <property>

     <name>dfs.namenode.rpc-address.bi.nn1</name>

     <value>mini01:</value>

   </property>

   <!-- nn1的http通信地址 -->

   <property>

     <name>dfs.namenode.http-address.bi.nn1</name>

     <value>mini01:</value>

   </property>

   <!-- nn2的RPC通信地址 -->

   <property>

     <name>dfs.namenode.rpc-address.bi.nn2</name>

     <value>mini02:</value>

   </property>

   <!-- nn2的http通信地址 -->

   <property>

     <name>dfs.namenode.http-address.bi.nn2</name>

     <value>mini02:</value>

   </property>

   <!-- nn3的RPC通信地址 -->

   <property>

     <name>dfs.namenode.rpc-address.dt.nn3</name>

     <value>mini03:</value>

   </property>

   <!-- nn3的http通信地址 -->

   <property>

     <name>dfs.namenode.http-address.dt.nn3</name>

     <value>mini03:</value>

   </property>

   <!-- nn4的RPC通信地址 -->

   <property>

     <name>dfs.namenode.rpc-address.dt.nn4</name>

     <value>mini04:</value>

   </property>

   <!-- nn4的http通信地址 -->

   <property>

     <name>dfs.namenode.http-address.dt.nn4</name>

     <value>mini04:</value>

   </property>

   <!-- 指定NameNode的edits元数据在JournalNode上的存放位置 -->

   <!--  在bi名称空间的两个namenode中用如下配置  # 在mini03和mini04中注释掉  -->

   <!--

   <property>

     <name>dfs.namenode.shared.edits.dir</name>

     <value>qjournal://mini05:8485;mini06:8485;mini07:8485/bi</value>

   </property>

    -->

   <!--  在dt名称空间的两个namenode中用如下配置  # 在mini01和mini02中注释掉  -->

   <property>

     <name>dfs.namenode.shared.edits.dir</name>

     <value>qjournal://mini05:8485;mini06:8485;mini07:8485/dt</value>

   </property>

   <!-- 指定JournalNode在本地磁盘存放数据的位置 -->

   <property>

     <name>dfs.journalnode.edits.dir</name>

     <value>/app/hadoop/journaldata</value>

   </property>

   <!-- 开启NameNode失败自动切换 -->

   <property>

     <name>dfs.ha.automatic-failover.enabled</name>

     <value>true</value>

   </property>

   <!-- 配置失败自动切换实现方式 -->

   <property>

     <name>dfs.client.failover.proxy.provider.bi</name>

     <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>

   </property>

   <property>

     <name>dfs.client.failover.proxy.provider.dt</name>

     <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>

   </property>

   <!-- 配置隔离机制方法，多个机制用换行分割，即每个机制暂用一行-->

   <!-- 其中shell(/bin/true) 表示可执行一个脚本  比如 shell(/app/yunwei/hadoop_fence.sh) -->

   <property>

     <name>dfs.ha.fencing.methods</name>

     <value>

       sshfence

       shell(/bin/true)

     </value>

   </property>

   <!-- 使用sshfence隔离机制时需要ssh免登陆 -->

   <property>

     <name>dfs.ha.fencing.ssh.private-key-files</name>

     <value>/app/.ssh/id_rsa</value>

   </property>

   <!-- 配置sshfence隔离机制超时时间 单位：毫秒 -->

   <property>

     <name>dfs.ha.fencing.ssh.connect-timeout</name>

     <value></value>

   </property>

 </configuration>

3.5. mapred-site.xml

 # 较上篇文章无改变

 [yun@mini01 hadoop]$ pwd

 /app/hadoop/etc/hadoop

 [yun@mini01 hadoop]$ cp -a mapred-site.xml.template mapred-site.xml

 [yun@mini01 hadoop]$ vim mapred-site.xml

 <?xml version="1.0"?>

 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

 <!--

   ……………………

 -->

 <!-- Put site-specific property overrides in this file. -->

 <configuration>

   <!-- 指定mr框架为yarn方式 -->

   <property>

     <name>mapreduce.framework.name</name>

     <value>yarn</value>

   </property>

 </configuration>

3.6. yarn-site.xml

 # 较上篇文章无改变

 [yun@mini01 hadoop]$ pwd

 /app/hadoop/etc/hadoop

 [yun@mini01 hadoop]$ vim yarn-site.xml

 <?xml version="1.0"?>

 <!--

   ……………………

 -->

 <configuration>

 <!-- Site specific YARN configuration properties -->

   <!-- 开启RM高可用 -->

   <property>

     <name>yarn.resourcemanager.ha.enabled</name>

     <value>true</value>

   </property>

   <!-- 指定RM的cluster id -->

   <property>

     <name>yarn.resourcemanager.cluster-id</name>

     <value>yrc</value>

   </property>

   <!-- 指定RM的名字 -->

   <property>

     <name>yarn.resourcemanager.ha.rm-ids</name>

     <value>rm1,rm2</value>

   </property>

   <!-- 分别指定RM的地址 -->

   <property>

     <name>yarn.resourcemanager.hostname.rm1</name>

     <value>mini03</value>

   </property>

   <property>

     <name>yarn.resourcemanager.hostname.rm2</name>

     <value>mini04</value>

   </property>

   <!-- 指定zk集群地址 -->

   <property>

     <name>yarn.resourcemanager.zk-address</name>

     <value>mini03:,mini04:,mini05:,mini06:,mini07:</value>

   </property>

   <!-- reduce 获取数据的方式 -->

   <property>

     <name>yarn.nodemanager.aux-services</name>

     <value>mapreduce_shuffle</value>

   </property>

 </configuration>

3.7. 修改slaves

slaves是指定子节点的位置，因为要在mini01上启动HDFS、在mini03启动yarn，所以mini01上的slaves文件指定的是datanode的位置，mini03上的slaves文件指定的是nodemanager的位置

 # 较上篇文章无改变

 [yun@mini01 hadoop]$ pwd

 /app/hadoop/etc/hadoop

 [yun@mini01 hadoop]$ vim slaves

 mini05

 mini06

 mini07

PS：改后配置后，将这些配置拷到其他Hadoop机器

4. 启动相关服务

注意：第一次启动时严格按照下面的步骤！！！！！！！

4.1. 启动zookeeper集群

前面已经启动了，这里就不说了

4.2. 启动journalnode

 # 根据规划在mini05、mini06、mini07 启动

 # 在第一次格式化的时候需要先启动journalnode   之后就不必了

 [yun@mini05 ~]$ hadoop-daemon.sh start journalnode  # 已经配置环境变量，所以不用进入到响应的目录

 starting journalnode, logging to /app/hadoop-2.7./logs/hadoop-yun-journalnode-mini05.out

 [yun@mini05 ~]$ jps

  QuorumPeerMain

  Jps

  JournalNode

4.3. 在bi的格式化

4.3.1. 格式化HDFS

 # 在mini01上执行命令

 [yun@mini01 ~]$ hdfs namenode -format -clusterID CID-8c9c1d4b-5aa3-4d12-b717-31a56884da7f   # 其中UUID可以在网上生成一个

 // :: INFO namenode.NameNode: STARTUP_MSG:

 /************************************************************

 STARTUP_MSG: Starting NameNode

 STARTUP_MSG:   host = mini01/10.0.0.111

 STARTUP_MSG:   args = [-format, -clusterID, CID-8c9c1d4b-5aa3-4d12-b717-31a56884da7f]

 STARTUP_MSG:   version = 2.7.6

 STARTUP_MSG:   classpath = ………………

 STARTUP_MSG:   build = Unknown -r Unknown; compiled by 'root' on 2018-06-08T08:30Z

 STARTUP_MSG:   java = 1.8.0_112

 ************************************************************/

 // :: INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]

 // :: INFO namenode.NameNode: createNameNode [-format, -clusterID, CID-8c9c1d4b-5aa3-4d12-b717-31a56884da7f]

 Formatting using clusterid: CID-8c9c1d4b-5aa3-4d12-b717-31a56884da7f

 // :: INFO namenode.FSNamesystem: No KeyProvider found.

 // :: INFO namenode.FSNamesystem: fsLock is fair: true

 // :: INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false

 // :: INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=

 // :: INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true

 // :: INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to :::00.000

 // :: INFO blockmanagement.BlockManager: The block deletion will start around  Jul  ::

 // :: INFO util.GSet: Computing capacity for map BlocksMap

 // :: INFO util.GSet: VM type       = -bit

 // :: INFO util.GSet: 2.0% max memory 966.7 MB = 19.3 MB

 // :: INFO util.GSet: capacity      = ^ =  entries

 // :: INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false

 // :: INFO blockmanagement.BlockManager: defaultReplication         =

 // :: INFO blockmanagement.BlockManager: maxReplication             =

 // :: INFO blockmanagement.BlockManager: minReplication             =

 // :: INFO blockmanagement.BlockManager: maxReplicationStreams      =

 // :: INFO blockmanagement.BlockManager: replicationRecheckInterval =

 // :: INFO blockmanagement.BlockManager: encryptDataTransfer        = false

 // :: INFO blockmanagement.BlockManager: maxNumBlocksToLog          =

 // :: INFO namenode.FSNamesystem: fsOwner             = yun (auth:SIMPLE)

 // :: INFO namenode.FSNamesystem: supergroup          = supergroup

 // :: INFO namenode.FSNamesystem: isPermissionEnabled = true

 // :: INFO namenode.FSNamesystem: Determined nameservice ID: bi

 // :: INFO namenode.FSNamesystem: HA Enabled: true

 // :: INFO namenode.FSNamesystem: Append Enabled: true

 // :: INFO util.GSet: Computing capacity for map INodeMap

 // :: INFO util.GSet: VM type       = -bit

 // :: INFO util.GSet: 1.0% max memory 966.7 MB = 9.7 MB

 // :: INFO util.GSet: capacity      = ^ =  entries

 // :: INFO namenode.FSDirectory: ACLs enabled? false

 // :: INFO namenode.FSDirectory: XAttrs enabled? true

 // :: INFO namenode.FSDirectory: Maximum size of an xattr:

 // :: INFO namenode.NameNode: Caching file names occuring more than  times

 // :: INFO util.GSet: Computing capacity for map cachedBlocks

 // :: INFO util.GSet: VM type       = -bit

 // :: INFO util.GSet: 0.25% max memory 966.7 MB = 2.4 MB

 // :: INFO util.GSet: capacity      = ^ =  entries

 // :: INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033

 // :: INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes =

 // :: INFO namenode.FSNamesystem: dfs.namenode.safemode.extension     =

 // :: INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets =

 // :: INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users =

 // :: INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = ,,

 // :: INFO namenode.FSNamesystem: Retry cache on namenode is enabled

 // :: INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is  millis

 // :: INFO util.GSet: Computing capacity for map NameNodeRetryCache

 // :: INFO util.GSet: VM type       = -bit

 // :: INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB

 // :: INFO util.GSet: capacity      = ^ =  entries

 // :: INFO namenode.FSImage: Allocated new BlockPoolId: BP--10.0.0.111-

 // :: INFO common.Storage: Storage directory /app/hadoop/tmp/dfs/name has been successfully formatted.

 // :: INFO namenode.FSImageFormatProtobuf: Saving image file /app/hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression

 // :: INFO namenode.FSImageFormatProtobuf: Image file /app/hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 of size  bytes saved in  seconds.

 // :: INFO namenode.NNStorageRetentionManager: Going to retain  images with txid >=

 // :: INFO util.ExitUtil: Exiting with status

 // :: INFO namenode.NameNode: SHUTDOWN_MSG:

 /************************************************************

 SHUTDOWN_MSG: Shutting down NameNode at mini01/10.0.0.111

 ************************************************************/

拷贝到mini02

 #格式化后会在根据core-site.xml中的hadoop.tmp.dir配置生成个文件，这里我配置的是/app/hadoop/tmp，然后将/app/hadoop/tmp拷贝到mini02的/app/hadoop/下。

 # 方法1：

 [yun@mini01 hadoop]$ pwd

 /app/hadoop

 [yun@mini01 hadoop]$ scp -r tmp/ yun@mini02:/app/hadoop

 VERSION                           %     .4KB/s   :

 seen_txid                         %         .0KB/s   :

 fsimage_0000000000000000000.md5   %       .7KB/s   :

 fsimage_0000000000000000000       %     .1KB/s   : 

 ##########################

 # 方法2：##也可以这样，建议hdfs namenode -bootstrapStandby  # 不过需要mini02的Hadoop起来才行

4.3.2. 格式化ZKFC

 # 在mini01上执行一次即可

 [yun@mini01 current]$ hdfs zkfc -formatZK

 // :: INFO tools.DFSZKFailoverController: Failover controller configured for NameNode NameNode at mini01/10.0.0.111:

 // :: INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.-, built on // : GMT

 // :: INFO zookeeper.ZooKeeper: Client environment:host.name=mini01

 // :: INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_112

 // :: INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation

 // :: INFO zookeeper.ZooKeeper: Client environment:java.home=/app/jdk1..0_112/jre

 // :: INFO zookeeper.ZooKeeper: Client environment:java.class.path=……………………

 // :: INFO zookeeper.ZooKeeper: Client environment:java.library.path=/app/hadoop-2.7./lib/native

 // :: INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp

 // :: INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>

 // :: INFO zookeeper.ZooKeeper: Client environment:os.name=Linux

 // :: INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64

 // :: INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.-.el7.x86_64

 // :: INFO zookeeper.ZooKeeper: Client environment:user.name=yun

 // :: INFO zookeeper.ZooKeeper: Client environment:user.home=/app

 // :: INFO zookeeper.ZooKeeper: Client environment:user.dir=/app/hadoop-2.7.

 // :: INFO zookeeper.ZooKeeper: Initiating client connection, connectString=mini03:,mini04:,mini05:,mini06:,mini07: sessionTimeout= watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@7f3b84b8

 // :: INFO zookeeper.ClientCnxn: Opening socket connection to server mini04/10.0.0.114:. Will not attempt to authenticate using SASL (unknown error)

 // :: INFO zookeeper.ClientCnxn: Socket connection established to mini04/10.0.0.114:, initiating session

 // :: INFO zookeeper.ClientCnxn: Session establishment complete on server mini04/10.0.0.114:, sessionid = 0x464538f087b0003, negotiated timeout =

 // :: INFO ha.ActiveStandbyElector: Session connected.

 // :: INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/bi in ZK.

 // :: INFO zookeeper.ZooKeeper: Session: 0x464538f087b0003 closed

 // :: INFO zookeeper.ClientCnxn: EventThread shut down

4.4. 在dt的格式化

4.4.1. 格式化HDFS

 # 在mini03上执行命令

 [yun@mini03 hadoop]$ hdfs namenode -format -clusterID CID-8c9c1d4b-5aa3-4d12-b717-31a56884da7f    # clusterID必须与bi的相同

 // :: INFO namenode.NameNode: STARTUP_MSG:

 /************************************************************

 STARTUP_MSG: Starting NameNode

 STARTUP_MSG:   host = mini03/10.0.0.113

 STARTUP_MSG:   args = [-format, -clusterID, CID-8c9c1d4b-5aa3-4d12-b717-31a56884da7f]

 STARTUP_MSG:   version = 2.7.6

 STARTUP_MSG:   classpath = ……………………

 STARTUP_MSG:   build = Unknown -r Unknown; compiled by 'root' on 2018-06-08T08:30Z

 STARTUP_MSG:   java = 1.8.0_112

 ************************************************************/

 // :: INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]

 // :: INFO namenode.NameNode: createNameNode [-format, -clusterID, CID-8c9c1d4b-5aa3-4d12-b717-31a56884da7f]

 Formatting using clusterid: CID-8c9c1d4b-5aa3-4d12-b717-31a56884da7f

 // :: INFO namenode.FSNamesystem: No KeyProvider found.

 // :: INFO namenode.FSNamesystem: fsLock is fair: true

 // :: INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false

 // :: INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=

 // :: INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true

 // :: INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to :::00.000

 // :: INFO blockmanagement.BlockManager: The block deletion will start around  Jul  ::

 // :: INFO util.GSet: Computing capacity for map BlocksMap

 // :: INFO util.GSet: VM type       = -bit

 // :: INFO util.GSet: 2.0% max memory 966.7 MB = 19.3 MB

 // :: INFO util.GSet: capacity      = ^ =  entries

 // :: INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false

 // :: INFO blockmanagement.BlockManager: defaultReplication         =

 // :: INFO blockmanagement.BlockManager: maxReplication             =

 // :: INFO blockmanagement.BlockManager: minReplication             =

 // :: INFO blockmanagement.BlockManager: maxReplicationStreams      =

 // :: INFO blockmanagement.BlockManager: replicationRecheckInterval =

 // :: INFO blockmanagement.BlockManager: encryptDataTransfer        = false

 // :: INFO blockmanagement.BlockManager: maxNumBlocksToLog          =

 // :: INFO namenode.FSNamesystem: fsOwner             = yun (auth:SIMPLE)

 // :: INFO namenode.FSNamesystem: supergroup          = supergroup

 // :: INFO namenode.FSNamesystem: isPermissionEnabled = true

 // :: INFO namenode.FSNamesystem: Determined nameservice ID: dt

 // :: INFO namenode.FSNamesystem: HA Enabled: true

 // :: INFO namenode.FSNamesystem: Append Enabled: true

 // :: INFO util.GSet: Computing capacity for map INodeMap

 // :: INFO util.GSet: VM type       = -bit

 // :: INFO util.GSet: 1.0% max memory 966.7 MB = 9.7 MB

 // :: INFO util.GSet: capacity      = ^ =  entries

 // :: INFO namenode.FSDirectory: ACLs enabled? false

 // :: INFO namenode.FSDirectory: XAttrs enabled? true

 // :: INFO namenode.FSDirectory: Maximum size of an xattr:

 // :: INFO namenode.NameNode: Caching file names occuring more than  times

 // :: INFO util.GSet: Computing capacity for map cachedBlocks

 // :: INFO util.GSet: VM type       = -bit

 // :: INFO util.GSet: 0.25% max memory 966.7 MB = 2.4 MB

 // :: INFO util.GSet: capacity      = ^ =  entries

 // :: INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033

 // :: INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes =

 // :: INFO namenode.FSNamesystem: dfs.namenode.safemode.extension     =

 // :: INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets =

 // :: INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users =

 // :: INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = ,,

 // :: INFO namenode.FSNamesystem: Retry cache on namenode is enabled

 // :: INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is  millis

 // :: INFO util.GSet: Computing capacity for map NameNodeRetryCache

 // :: INFO util.GSet: VM type       = -bit

 // :: INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB

 // :: INFO util.GSet: capacity      = ^ =  entries

 // :: INFO namenode.FSImage: Allocated new BlockPoolId: BP--10.0.0.113-

 // :: INFO common.Storage: Storage directory /app/hadoop/tmp/dfs/name has been successfully formatted.

 // :: INFO namenode.FSImageFormatProtobuf: Saving image file /app/hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression

 // :: INFO namenode.FSImageFormatProtobuf: Image file /app/hadoop/tmp/dfs/name/current/fsimage.ckpt_0000000000000000000 of size  bytes saved in  seconds.

 // :: INFO namenode.NNStorageRetentionManager: Going to retain  images with txid >=

 // :: INFO util.ExitUtil: Exiting with status

 // :: INFO namenode.NameNode: SHUTDOWN_MSG:

 /************************************************************

 SHUTDOWN_MSG: Shutting down NameNode at mini03/10.0.0.113

 ************************************************************/

拷贝到mini04

 #格式化后会在根据core-site.xml中的hadoop.tmp.dir配置生成个文件，这里我配置的是/app/hadoop/tmp，然后将/app/hadoop/tmp拷贝到mini04的/app/hadoop/下。

 # 方法1：

 [yun@mini03 hadoop]$ pwd

 /app/hadoop

 [yun@mini03 hadoop]$ scp -r tmp/ yun@mini04:/app/hadoop

 VERSION                           %     .4KB/s   :

 seen_txid                         %         .0KB/s   :

 fsimage_0000000000000000000.md5   %       .7KB/s   :

 fsimage_0000000000000000000       %     .1KB/s   : 

 ##########################

 # 方法2：##也可以这样，建议hdfs namenode -bootstrapStandby  # 不过需要mini04的Hadoop起来才行

4.4.2. 格式化ZKFC

 # 在mini03上执行一次即可

 [yun@mini03 hadoop]$ hdfs zkfc -formatZK

 // :: INFO tools.DFSZKFailoverController: Failover controller configured for NameNode NameNode at mini03/10.0.0.113:

 // :: INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.-, built on // : GMT

 // :: INFO zookeeper.ZooKeeper: Client environment:host.name=mini03

 // :: INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_112

 // :: INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation

 // :: INFO zookeeper.ZooKeeper: Client environment:java.home=/app/jdk1..0_112/jre

 // :: INFO zookeeper.ZooKeeper: Client environment:java.class.path=……………………

 // :: INFO zookeeper.ZooKeeper: Client environment:java.library.path=/app/hadoop-2.7./lib/native

 // :: INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp

 // :: INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>

 // :: INFO zookeeper.ZooKeeper: Client environment:os.name=Linux

 // :: INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64

 // :: INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.-.el7.x86_64

 // :: INFO zookeeper.ZooKeeper: Client environment:user.name=yun

 // :: INFO zookeeper.ZooKeeper: Client environment:user.home=/app

 // :: INFO zookeeper.ZooKeeper: Client environment:user.dir=/app/hadoop-2.7.

 // :: INFO zookeeper.ZooKeeper: Initiating client connection, connectString=mini03:,mini04:,mini05:,mini06:,mini07: sessionTimeout= watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@7f3b84b8

 // :: INFO zookeeper.ClientCnxn: Opening socket connection to server mini05/10.0.0.115:. Will not attempt to authenticate using SASL (unknown error)

 // :: INFO zookeeper.ClientCnxn: Socket connection established to mini05/10.0.0.115:, initiating session

 // :: INFO zookeeper.ClientCnxn: Session establishment complete on server mini05/10.0.0.115:, sessionid = 0x56455467ae50003, negotiated timeout =

 // :: INFO ha.ActiveStandbyElector: Session connected.

 // :: INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/dt in ZK.

 // :: INFO zookeeper.ZooKeeper: Session: 0x56455467ae50003 closed

 // :: INFO zookeeper.ClientCnxn: EventThread shut down

4.5. 启动HDFS

 # 在mini01操作一次即可

 [yun@mini01 hadoop]$ start-dfs.sh

 Starting namenodes on [mini01 mini02 mini03 mini04]

 mini01: starting namenode, logging to /app/hadoop-2.7./logs/hadoop-yun-namenode-mini01.out

 mini02: starting namenode, logging to /app/hadoop-2.7./logs/hadoop-yun-namenode-mini02.out

 mini04: starting namenode, logging to /app/hadoop-2.7./logs/hadoop-yun-namenode-mini04.out

 mini03: starting namenode, logging to /app/hadoop-2.7./logs/hadoop-yun-namenode-mini03.out

 mini06: starting datanode, logging to /app/hadoop-2.7./logs/hadoop-yun-datanode-mini06.out

 mini07: starting datanode, logging to /app/hadoop-2.7./logs/hadoop-yun-datanode-mini07.out

 mini05: starting datanode, logging to /app/hadoop-2.7./logs/hadoop-yun-datanode-mini05.out

 Starting journal nodes [mini05 mini06 mini07]

 mini05: journalnode running as process . Stop it first.

 mini07: journalnode running as process . Stop it first.

 mini06: journalnode running as process . Stop it first.

 Starting ZK Failover Controllers on NN hosts [mini01 mini02 mini03 mini04]

 mini04: starting zkfc, logging to /app/hadoop-2.7./logs/hadoop-yun-zkfc-mini04.out

 mini01: starting zkfc, logging to /app/hadoop-2.7./logs/hadoop-yun-zkfc-mini01.out

 mini02: starting zkfc, logging to /app/hadoop-2.7./logs/hadoop-yun-zkfc-mini02.out

 mini03: starting zkfc, logging to /app/hadoop-2.7./logs/hadoop-yun-zkfc-mini03.out

4.6. 启动YARN

 #####注意#####：是在mini03上执行start-yarn.sh，把namenode和resourcemanager分开是因为性能问题

 # 因为他们都要占用大量资源，所以把他们分开了，他们分开了就要分别在不同的机器上启动

 [yun@mini03 ~]$ start-yarn.sh

 starting yarn daemons

 starting resourcemanager, logging to /app/hadoop-2.7./logs/yarn-yun-resourcemanager-mini03.out

 mini07: starting nodemanager, logging to /app/hadoop-2.7./logs/yarn-yun-nodemanager-mini07.out

 mini06: starting nodemanager, logging to /app/hadoop-2.7./logs/yarn-yun-nodemanager-mini06.out

 mini05: starting nodemanager, logging to /app/hadoop-2.7./logs/yarn-yun-nodemanager-mini05.out

 ################################

 # 在mini04启动 resourcemanager

 [yun@mini04 ~]$ yarn-daemon.sh start resourcemanager  # 也可用start-yarn.sh

 starting resourcemanager, logging to /app/hadoop-2.7./logs/yarn-yun-resourcemanager-mini04.out

4.7. 启动说明

 # 第一次启动的时候请严格按照上面的步骤【第一次涉及格式化问题】

 # 第二次以及之后，步骤为： 启动zookeeper、HDFS、YARN

5. 浏览器访问

5.1. bi访问

 http://mini01:50070

 http://mini02:50070

5.2. dt访问

 http://mini03:50070

 http://mini04:50070

6. 简单操作

6.1. HDFS创建目录与上传

 [yun@mini02 software]$ pwd

 /app/software

 [yun@mini02 software]$ ll

 total

 -rw-r--r--  yun yun  Jun   : CentOS-.4_hadoop-2.7..tar.gz

 -rw-r--r--  yun yun  May  : eclipse-jee-oxygen-3a-win32-x86_64.zip

 -rw-r--r--  yun yun  Oct    jdk1..0_112.tar.gz

 -rw-rw-r--  yun yun          Jul   : test

 [yun@mini02 software]$ hadoop fs -ls /   # 通过命令行可见，是进行了逻辑分区的

 Found  items

 -r-xr-xr-x   - yun yun           -- : /bi

 -r-xr-xr-x   - yun yun           -- : /dt

 [yun@mini02 software]$ hadoop fs -mkdir /bi/software   # 在bi创建目录

 [yun@mini02 software]$

 [yun@mini02 software]$ hadoop fs -mkdir /dt/software   # 在dt创建目录

 [yun@mini02 software]$

 [yun@mini02 software]$ hadoop fs -put jdk1..0_112.tar.gz /bi/software # 上传文件到bi

 [yun@mini02 software]$ hadoop fs -put eclipse-jee-oxygen-3a-win32-x86_64.zip /dt/software # 上传文件到dt

 [yun@mini02 software]$

 [yun@mini02 software]$

 [yun@mini02 software]$ hadoop fs -ls /bi/software

 Found  items

 -rw-r--r--    yun supergroup   -- : /bi/software/jdk1..0_112.tar.gz

 [yun@mini02 software]$

 [yun@mini02 software]$

 [yun@mini02 software]$ hadoop fs -ls /dt/software

 Found  items

 -rw-r--r--    yun supergroup   -- : /dt/software/eclipse-jee-oxygen-3a-win32-x86_64.zip

6.2. Datanode目录结构

 [yun@mini05 current]$ pwd

 /app/hadoop/tmp/dfs/data/current

 [yun@mini05 current]$ ll

 total

 drwx------  yun yun   Jul   : BP--10.0.0.111-

 drwx------  yun yun   Jul   : BP--10.0.0.113-

 -rw-rw-r--  yun yun  Jul   : VERSION

 [yun@mini05 current]$ tree

 .

 ├── BP--10.0.0.111-

 │   ├── current

 │   │   ├── finalized

 │   │   │   └── subdir0

 │   │   │       └── subdir0

 │   │   │           ├── blk_1073741825

 │   │   │           ├── blk_1073741825_1001.meta

 │   │   │           ├── blk_1073741826

 │   │   │           └── blk_1073741826_1002.meta

 │   │   ├── rbw

 │   │   └── VERSION

 │   ├── scanner.cursor

 │   └── tmp

 ├── BP--10.0.0.113-

 │   ├── current

 │   │   ├── finalized

 │   │   │   └── subdir0

 │   │   │       └── subdir0

 │   │   │           ├── blk_1073741825

 │   │   │           ├── blk_1073741825_1001.meta

 │   │   │           ├── blk_1073741826

 │   │   │           ├── blk_1073741826_1002.meta

 │   │   │           ├── blk_1073741827

 │   │   │           └── blk_1073741827_1003.meta

 │   │   ├── rbw

 │   │   └── VERSION

 │   ├── scanner.cursor

 │   └── tmp

 └── VERSION

  directories,  files

Hadoop2.7.6_08_Federation联邦机制的更多相关文章

Hadoop记录-Federation联邦机制
在Hadoop2.0之前,HDFS的单NameNode设计带来诸多问题: 单点故障.内存受限,制约集群扩展性和缺乏隔离机制(不同业务使用同一个NameNode导致业务相互影响)等为了解决这些问题, ...
2-10 就业课(2.0)-oozie：11、hadoop的federation（联邦机制，了解一下）
==================================================== Hadoop Federation 背景概述单NameNode的架构使得HDFS在集群扩展性 ...
HDFS 10 - HDFS 的联邦机制（Federation 机制）
目录 1 - 为什么需要联邦 2 - Federation 架构设计 3 HDFS Federation 的不足版权声明 1 - 为什么需要联邦单 NameNode 的架构存在的问题:当集群中数据 ...
Hadoop_HDFS架构和HA机制
Hadoop学习笔记总结 01.HDFS架构 1. NameNode和ResourceManager NameNode负责HDFS,从节点是DataNode:ResourceManager负责MapR ...
Cloudera Hadoop 5& Hadoop高阶管理及调优课程(CDH5,Hadoop2.0,HA,安全,管理,调优)
1.课程环境本课程涉及的技术产品及相关版本: 技术版本 Linux CentOS 6.5 Java 1.7 Hadoop2.0 2.6.0 Hadoop1.0 1.2.1 Zookeeper 3. ...
Prometheus监控学习笔记之prometheus的federation机制
0x00 概述有时候对于一个公司,k8s集群或是所谓的caas只是整个技术体系的一部分,往往这个时候监控系统不仅仅要k8s集群以及k8s中部署的应用,而且要监控传统部署的项目.也就是说整个监控系统不 ...
联邦学习（Federated Learning）
联邦学习简介联邦学习(Federated Learning)是一种新兴的人工智能基础技术,在 2016 年由谷歌最先提出,原本用于解决安卓手机终端用户在本地更新模型的问题,其设计目标是 ...
Federal Learning（联邦学习）认知
本人是学生党,同时也是小菜鸡一枚,撞运气有机会能够给老师当项目助理,在这个过程中肯定会学到一些有趣的知识,就在此平台上记录一下,在知识点方面有不对的还请各位指正. What(什么是联邦学习?) 联邦学 ...
大数据学习笔记4 - Hadoop的优化与发展（Hadoop 2.0）
前面介绍了Hadoop核心组件HDFS和MapReduce,Hadoop发展之初在架构设计和应用性能方面仍然存在不足,Hadoop的优化与发展一方面体现在两个核心组件的架构设计改进,一方面体现在Had ...

随机推荐

Xdebug调试环境配置
首先我们需要下载Xdebug 务必根据安装的PHP版本,选择合适的Xdebug版本,由于我是在Windows环境下安装PHP的,选择下载Windows版本的PHP 5.3 VC9 TS (32 bit ...
Java字符串和容器
String Java.lang.String是Java的字符串类. Srting是一个不可变对象,所有对String修改的操作都需要构造新的String实例. String可以由char数组或字符串 ...
python argparse(参数解析模块)
这是一个参数解析,可以用它快捷的为你的程序生成参数相关功能 import argparse(导入程序参数模块) # 创建argparse对象,并将产品简要说明加入show = '程序说明' ===&g ...
C#添加IIS站点
利用IIS7自带类库管理IIS现在变的更强大更方便,而完全可以不需要用DirecotryEntry这个类了(乐博网中很多.net管理iis6.0的文章都用到了DirecotryEntry这个类 ),M ...
Oracle索引失效原因及解决方法
一.Oracle索引失效的原因 1使用否定关键字 !=, <> ,not in,not exist select * fromdrama where id <> 1,Mysql ...
css引入页面的三种方法
1.内联式:直接在标签上写样式 <!DOCTYPE html> <html lang="en"> <head> <meta charset ...
CSS3景深-perspective
3D视图正方体: <!DOCTYPE html> <html lang="en"> <head> <meta charset=" ...
【读书笔记】iOS-对iOS应用进行模糊测试
一,模糊测试,是指通过反复向待测应用发送畸形的数据,对应用进行动态测试的过程. 二,模糊测试,也称动态分析,是一种构造非法输入并将其提供给应用,以期让应用暴露出某些安全问题的艺术和科学. 参考资料:& ...
使用CSS兄弟选择器完成复杂垂直边距（vertical margins）的设计
-------------------sibling选择器如何在完成复杂设计要求的同时,保持CSS可读这是web前端开发过程中开始简单逐步变的复杂的例子之一:将一篇文章中的所有元素应用垂直边距(ve ...
二层协议--LLDP协议总结
二层协议--LLDP协议总结,待完善.

Hadoop2.7.6_08_Federation联邦机制