hadoop 根据secondary namenode恢复namenode
refer to
http://www.cnblogs.com/Richardzhu/p/3435989.html
http://blog.csdn.net/wuzhilon88/article/details/49506873
方法一、使用namespaceID
1、在namenode节点上，将dfs.name.dir指定的目录中(这里是name目录)的内容情况，以此来模拟故障发生。
1 [hadoop@node1 name]$ ls
2 current image in_use.lock
3 [hadoop@node1 name]$ rm -rf *
2、将集群关闭后，再重启我们看到namenode守护进程消失。
1 [hadoop@node1 name]$ stop-all.sh
2 stopping jobtracker
3 192.168.1.152: stopping tasktracker
4 192.168.1.153: stopping tasktracker
5 stopping namenode
6 192.168.1.152: stopping datanode
7 192.168.1.153: stopping datanode
8 192.168.1.152: stopping secondarynamenode
9 [hadoop@node1 name]$ start-all.sh
10 starting namenode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-namenode-node1.out
11 192.168.1.152: starting datanode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-datanode-node2.out
12 192.168.1.153: starting datanode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-datanode-node3.out
13 192.168.1.152: starting secondarynamenode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-node2.out
14 starting jobtracker, logging to /app/hadoop/bin/../logs/hadoop-hadoop-jobtracker-node1.out
15 192.168.1.152: starting tasktracker, logging to /app/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node2.out
16 192.168.1.153: starting tasktracker, logging to /app/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node3.out
17 [hadoop@node1 name]$ jps
18 31942 Jps
19 31872 JobTracker
而且namenode的日志中有报错：
1 2013-11-14 06:19:59,172 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG:
2 /************************************************************
3 STARTUP_MSG: Starting NameNode
4 STARTUP_MSG: host = node1/192.168.1.151
5 STARTUP_MSG: args = []
6 STARTUP_MSG: version = 0.20.2
7 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
8 ************************************************************/
9 2013-11-14 06:19:59,395 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=NameNode, port=9000
10 2013-11-14 06:19:59,400 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at: node1.com/192.168.1.151:9000
11 2013-11-14 06:19:59,403 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=NameNode, sessionId=null
12 2013-11-14 06:19:59,407 INFO org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics: Initializing NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullContext
13 2013-11-14 06:19:59,557 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=hadoop,hadoop
14 2013-11-14 06:19:59,558 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
15 2013-11-14 06:19:59,558 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled=true
16 2013-11-14 06:19:59,568 INFO org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics: Initializing FSNamesystemMetrics using context object:org.apache.hadoop.metrics.spi.NullContext
17 2013-11-14 06:19:59,569 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemStatusMBean
18 2013-11-14 06:19:59,654 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
19 java.io.IOException: NameNode is not formatted.
20 at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
21 at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
22 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
23 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
24 at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
25 at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
26 at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
27 at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
28 2013-11-14 06:19:59,658 INFO org.apache.hadoop.ipc.Server: Stopping server on 9000
29 2013-11-14 06:19:59,663 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException: NameNode is not formatted.
30 at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
31 at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
32 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
33 at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
34 at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
35 at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
36 at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
37 at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
38
39 2013-11-14 06:19:59,664 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
40 /************************************************************
41 SHUTDOWN_MSG: Shutting down NameNode at node1/192.168.1.151
42 ************************************************************/
3、查看HDFS的文件失败：
1 [hadoop@node1 name]$ hadoop dfs -ls /user/hive/warehouse
2 13/11/14 06:21:06 INFO ipc.Client: Retrying connect to server: node1/192.168.1.151:9000. Already tried 0 time(s).
3 13/11/14 06:21:07 INFO ipc.Client: Retrying connect to server: node1/192.168.1.151:9000. Already tried 1 time(s).
4 13/11/14 06:21:08 INFO ipc.Client: Retrying connect to server: node1/192.168.1.151:9000. Already tried 2 time(s).
5 13/11/14 06:21:09 INFO ipc.Client: Retrying connect to server: node1/192.168.1.151:9000. Already tried 3 time(s).
4、关闭集群，格式化namenode：
1 [hadoop@node1 name]$ stop-all.sh
2 stopping jobtracker
3 192.168.1.152: stopping tasktracker
4 192.168.1.153: stopping tasktracker
5 no namenode to stop
6 192.168.1.152: stopping datanode
7 192.168.1.153: stopping datanode
8 192.168.1.152: stopping secondarynamenode
9 [hadoop@node1 name]$ hadoop namenode -format
10 13/11/14 06:21:37 INFO namenode.NameNode: STARTUP_MSG:
11 /************************************************************
12 STARTUP_MSG: Starting NameNode
13 STARTUP_MSG: host = node1/192.168.1.151
14 STARTUP_MSG: args = [-format]
15 STARTUP_MSG: version = 0.20.2
16 STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
17 ************************************************************/
18 Re-format filesystem in /app/user/hdfs/name ? (Y or N) Y
19 13/11/14 06:21:39 INFO namenode.FSNamesystem: fsOwner=hadoop,hadoop
20 13/11/14 06:21:39 INFO namenode.FSNamesystem: supergroup=supergroup
21 13/11/14 06:21:39 INFO namenode.FSNamesystem: isPermissionEnabled=true
22 13/11/14 06:21:39 INFO common.Storage: Image file of size 96 saved in 0 seconds.
23 13/11/14 06:21:39 INFO common.Storage: Storage directory /app/user/hdfs/name has been successfully formatted.
24 13/11/14 06:21:39 INFO namenode.NameNode: SHUTDOWN_MSG:
25 /************************************************************
26 SHUTDOWN_MSG: Shutting down NameNode at node1/192.168.1.151
27 ************************************************************/
5、从任意datanode中获取namenode格式化之前namespaceID并修改namenode的namespaceID跟datanode一致：
1 [hadoop@node2 current]$ cat VERSION
2 #Thu Nov 14 02:27:10 CST 2013
3 namespaceID=2062292356
4 storageID=DS-107813142-192.168.1.152-50010-1379339943465
5 cTime=0
6 storageType=DATA_NODE
7 layoutVersion=-18
8 [hadoop@node2 current]$ pwd
9 /app/user/hdfs/data/current
10 ----修改namenode的namespaceID----
[hadoop@node1 current]$ cat VERSION
11 #Thu Nov 14 06:29:31 CST 2013
12 namespaceID=2062292356
13 cTime=0
14 storageType=NAME_NODE
15 layoutVersion=-18
6、删除新的namenode的fsimage文件：
1 [hadoop@node1 current]$ ll
2 total 16
3 -rw-rw-r-- 1 hadoop hadoop 4 Nov 14 06:21 edits
4 -rw-rw-r-- 1 hadoop hadoop 96 Nov 14 06:21 fsimage
5 -rw-rw-r-- 1 hadoop hadoop 8 Nov 14 06:21 fstime
6 -rw-rw-r-- 1 hadoop hadoop 101 Nov 14 06:22 VERSION
7 [hadoop@node1 current]$ rm fsimage
7、从Secondarynamenode拷贝fsimage到Namenode的current目录下：
[hadoop@node2 current]$ ll
total 16
-rw-rw-r-- 1 hadoop hadoop 4 Nov 14 05:38 edits
-rw-rw-r-- 1 hadoop hadoop 2410 Nov 14 05:38 fsimage
-rw-rw-r-- 1 hadoop hadoop 8 Nov 14 05:38 fstime
-rw-rw-r-- 1 hadoop hadoop 101 Nov 14 05:38 VERSION
[hadoop@node2 current]$ scp fsimage node1:/app/user/hdfs/name/current
The authenticity of host 'node1 (192.168.1.151)' can't be established.
RSA key fingerprint is ca:9a:7e:19:ee:a1:35:44:7e:9d:d4:09:5c:fc:c5:0a.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node1,192.168.1.151' (RSA) to the list of known hosts.
fsimage 100% 2410 2.4KB/s 00:00
8、重启集群：
[hadoop@node1 current]$ start-all.sh
starting namenode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-namenode-node1.out
192.168.1.152: starting datanode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-datanode-node2.out
192.168.1.153: starting datanode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-datanode-node3.out
192.168.1.152: starting secondarynamenode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-node2.out
starting jobtracker, logging to /app/hadoop/bin/../logs/hadoop-hadoop-jobtracker-node1.out
192.168.1.152: starting tasktracker, logging to /app/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node2.out
192.168.1.153: starting tasktracker, logging to /app/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node3.out
[hadoop@node1 current]$ jps
32486 Jps
32419 JobTracker
32271 NameNode
9、验证数据的完整性：
1 [hadoop@node1 current]$ hadoop dfs -ls /user/hive/warehouse
2 Found 8 items
3 drwxr-xr-x - hadoop supergroup 0 2013-10-17 16:18 /user/hive/warehouse/echo
4 drwxr-xr-x - hadoop supergroup 0 2013-10-28 13:48 /user/hive/warehouse/jack
5 drwxr-xr-x - hadoop supergroup 0 2013-09-18 15:54 /user/hive/warehouse/table4
6 drwxr-xr-x - hadoop supergroup 0 2013-09-18 15:53 /user/hive/warehouse/table5
7 drwxr-xr-x - hadoop supergroup 0 2013-09-18 15:48 /user/hive/warehouse/test
8 drwxr-xr-x - hadoop supergroup 0 2013-10-25 14:50 /user/hive/warehouse/test1
9 drwxr-xr-x - hadoop supergroup 0 2013-10-25 14:52 /user/hive/warehouse/test2
10 drwxr-xr-x - hadoop supergroup 0 2013-10-25 14:30 /user/hive/warehouse/test3
11
12 [hadoop@node3 conf]$ hive
13
14 Logging initialized using configuration in jar:file:/app/hive/lib/hive-common-0.11.0.jar!/hive-log4j.properties
15 Hive history file=/tmp/hadoop/hive_job_log_hadoop_7451@node3_201311111325_424288589.txt
16 hive> show tables;
17 OK
18 echo
19 jack
20 table4
21 table5
22 test
23 test1
24 test2
25 test3
26 Time taken: 27.589 seconds, Fetched: 8 row(s)
27 hive> select * from table4;
28 OK
29 NULL NULL NULL
30 1 1 5
31 2 4 5
32 3 4 5
33 4 5 6
34 5 6 7
35 6 1 5
36 7 5 6
37 8 3 6
38 NULL NULL NULL
39 Time taken: 2.124 seconds, Fetched: 10 row(s)
之前里面的数据没有丢失。
方法二：使用hadoop namenode -importCheckpoint
1、删除name目录：
1 [hadoop@node1 hdfs]$ rm -rf name
2、关闭集群，从secondarynamenode拷贝namesecondary目录到dfs.name.dir：
[hadoop@node2 hdfs]$ scp -r namesecondary node1:/app/user/hdfs/
fsimage 100% 157 0.2KB/s 00:00
fstime 100% 8 0.0KB/s 00:00
fsimage 100% 2410 2.4KB/s 00:00
VERSION 100% 101 0.1KB/s 00:00
edits 100% 4 0.0KB/s 00:00
fstime 100% 8 0.0KB/s 00:00
fsimage 100% 2410 2.4KB/s 00:00
VERSION 100% 101 0.1KB/s 00:00
edits 100% 4 0.0KB/s 00:00
3、在namenode节点上执行hadoop namenode -importCheckpoint
[hadoop@node1 hdfs]$ hadoop namenode -importCheckpoint
13/11/14 07:24:20 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = node1/192.168.1.151
STARTUP_MSG: args = [-importCheckpoint]
STARTUP_MSG: version = 0.20.2
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
13/11/14 07:24:20 INFO metrics.RpcMetrics: Initializing RPC Metrics with hostName=NameNode, port=9000
13/11/14 07:24:20 INFO namenode.NameNode: Namenode up at: node1.com/192.168.1.151:9000
13/11/14 07:24:20 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=NameNode, sessionId=null
13/11/14 07:24:20 INFO metrics.NameNodeMetrics: Initializing NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullContext
13/11/14 07:24:21 INFO namenode.FSNamesystem: fsOwner=hadoop,hadoop
13/11/14 07:24:21 INFO namenode.FSNamesystem: supergroup=supergroup
13/11/14 07:24:21 INFO namenode.FSNamesystem: isPermissionEnabled=true
13/11/14 07:24:21 INFO metrics.FSNamesystemMetrics: Initializing FSNamesystemMetrics using context object:org.apache.hadoop.metrics.spi.NullContext
13/11/14 07:24:21 INFO namenode.FSNamesystem: Registered FSNamesystemStatusMBean
13/11/14 07:24:21 INFO common.Storage: Storage directory /app/user/hdfs/name is not formatted.
13/11/14 07:24:21 INFO common.Storage: Formatting ...
13/11/14 07:24:21 INFO common.Storage: Number of files = 26
13/11/14 07:24:21 INFO common.Storage: Number of files under construction = 0
13/11/14 07:24:21 INFO common.Storage: Image file of size 2410 loaded in 0 seconds.
13/11/14 07:24:21 INFO common.Storage: Edits file /app/user/hdfs/namesecondary/current/edits of size 4 edits # 0 loaded in 0 seconds.
13/11/14 07:24:21 INFO common.Storage: Image file of size 2410 saved in 0 seconds.
13/11/14 07:24:21 INFO common.Storage: Image file of size 2410 saved in 0 seconds.
13/11/14 07:24:21 INFO namenode.FSNamesystem: Number of transactions: 0 Total time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
13/11/14 07:24:21 INFO namenode.FSNamesystem: Finished loading FSImage in 252 msecs
13/11/14 07:24:21 INFO hdfs.StateChange: STATE* Safe mode ON.
The ratio of reported blocks 0.0000 has not reached the threshold 0.9990. Safe mode will be turned off automatically.
13/11/14 07:24:21 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
13/11/14 07:24:21 INFO http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50070
13/11/14 07:24:21 INFO http.HttpServer: listener.getLocalPort() returned 50070 webServer.getConnectors()[0].getLocalPort() returned 50070
13/11/14 07:24:21 INFO http.HttpServer: Jetty bound to port 50070
13/11/14 07:24:21 INFO mortbay.log: jetty-6.1.14
13/11/14 07:24:21 INFO mortbay.log: Started SelectChannelConnector@node1.com:50070
13/11/14 07:24:21 INFO namenode.NameNode: Web-server up at: node1.com:50070
13/11/14 07:24:21 INFO ipc.Server: IPC Server Responder: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server listener on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 0 on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 1 on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 2 on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 3 on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 4 on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 5 on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 6 on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 9 on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 7 on 9000: starting
13/11/14 07:24:21 INFO ipc.Server: IPC Server handler 8 on 9000: starting
13/11/14 07:37:05 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at node1/192.168.1.151
************************************************************/
[hadoop@node1 current]$ start-all.sh
starting namenode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-namenode-node1.out
192.168.1.152: starting datanode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-datanode-node2.out
192.168.1.153: starting datanode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-datanode-node3.out
192.168.1.152: starting secondarynamenode, logging to /app/hadoop/bin/../logs/hadoop-hadoop-secondarynamenode-node2.out
starting jobtracker, logging to /app/hadoop/bin/../logs/hadoop-hadoop-jobtracker-node1.out
192.168.1.152: starting tasktracker, logging to /app/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node2.out
192.168.1.153: starting tasktracker, logging to /app/hadoop/bin/../logs/hadoop-hadoop-tasktracker-node3.out
[hadoop@node1 current]$ jps
1027 JobTracker
1121 Jps
879 NameNode
4、验证数据的完整性：
1 [hadoop@node3 conf]$ hive
2
3 Logging initialized using configuration in jar:file:/app/hive/lib/hive-common-0.11.0.jar!/hive-log4j.properties
4 Hive history file=/tmp/hadoop/hive_job_log_hadoop_8383@node3_201311111443_2018635710.txt
5 hive> select * from table4;
6 OK
7 NULL NULL NULL
8 1 1 5
9 2 4 5
10 3 4 5
11 4 5 6
12 5 6 7
13 6 1 5
14 7 5 6
15 8 3 6
16 NULL NULL NULL
17 Time taken: 3.081 seconds, Fetched: 10 row(s)
总结：
注意：恢复的namenode中secondarynamenode的最近一次check到故障发生这段时间的内容将丢失，所以fs.checkpoint.period参数值在实际设定中要尽可能的权衡。并且也时常备份secondarynamenode节点中的内容，因为scondarynamenode也是单点的，以防发生故障。
补充说明：如果是用新的节点来恢复namenode，则要注意
1、新节点的Linux环境，目录结构，环境变量等等配置需要跟原来的namenode一模一样，包括conf目录下的所有文件配置。
2、新namenode的主机名要与原namenode保持一致，如果是重新命名主机名的话，则需要批量替换datanode和secondarynamenode的hosts文件，并且重新配置以下文件的部分core-site.xml文件中的fs.default.name
hdfs-site.xml文件中的dfs.http.address(secondarynamenode节点上)
mapred-site.xml文件中的mapred.job.tracker(如果jobtracker与namenode在同一个机器上，一般都是同一台机器上)。
hadoop 根据secondary namenode恢复namenode的更多相关文章
- hadoop 根据SecondaryNameNode恢复Namenode
1.修改conf/core-site.xml 增加 <property> <name>fs.checkpoint.period</name> <value&g ...
- 从 secondarynamenode 中恢复 namenode
1.修改 conf/core-site.xml,增加 Xml代码 <property> <name>fs.checkpoint.period</name> < ...
- hadoop 的HDFS 的 standby namenode无法启动事故处理
standby namenode无法启动 现象:线上使用的2.5.0-cdh5.3.2版本Hadoop,开启了了NameNode HA,HA采用QJM方式.hadoop的集群的namenode的sta ...
- Apache Hadoop集群安装（NameNode HA + SPARK + 机架感知）
1.主机规划 序号 主机名 IP地址 角色 1 nn-1 192.168.9.21 NameNode.mr-jobhistory.zookeeper.JournalNode 2 nn-2 ).HA的集 ...
- Apache Hadoop集群安装（NameNode HA + YARN HA + SPARK + 机架感知）
1.主机规划 序号 主机名 IP地址 角色 1 nn-1 192.168.9.21 NameNode.mr-jobhistory.zookeeper.JournalNode 2 nn-2 192.16 ...
- hadoop启动后jps没有namenode(转)
hadoop启动后jps没有namenode 一般都是由于两次或两次以上格式化NameNode造成的,有两种方法可以解决: 1.删除DataNode的所有资料 2.修改每个DataNode的names ...
- Hadoop（五）—— HDFS NameNode、DataNode工作机制
一.NN与2NN工作机制 NameNode(NN) 1.当HDFS启动时,会加载日志(edits)和镜像文件(fsImage)到内存中. 2-4.当元数据的增删改查请求进来时,NameNode会先将操 ...
- hadoop的hdfs中的namenode和datanode知识总结
一,NameNode: 1, Namenode是中心服务器,单一节点(简化系统的设计和实现),负责管理文件系统的名称空间(namespace)以及客户端对文件的访问. 2, 文件操作,Namenod ...
- Hadoop之Secondary NameNode
NameNode存储文件系统的变化作为log追加在本地的一个文件里:这个文件是edits.当一个NameNode启动时,它从一个映像文件:FsImage,读取HDFS的状态,使用来自edits日志文件 ...
随机推荐
- .aspx 页面引用命名空间
一.单个页面引用: <%@ Import Namespace="" %> 二.所有页面引用,Web.config配置如下: <system.web> < ...
- Java 分页之最简单的算法
分页实现有很多方式,如jQuery自带框架pagination或在java封装一个类pager等. 下写一个简单易懂的分页算法 逻辑: // 步骤1:设置每页页数大小 long pageS ...
- declare @t table
DECLARE @t TABLE(date char(21))INSERT @t SELECT '1900-1-1 00:00:00.000'INSERT @t SELECT '1900-1-1 00 ...
- 在Java中怎样高效的推断数组中是否包括某个元素
来自 http://www.hollischuang.com/archives/1269? 怎样检查一个数组(无序)是否包括一个特定的值?这是一个在Java中经经常使用到的并且非常实用的操作.同一时候 ...
- vim 基本常用设置
1. :set number 设置行号 2. :set relativenumber 设置相对行号 3. :set hlsearch 设置搜索高亮 4. :set noh 关闭搜索高亮 ...
- Fiddler 默认不能抓取页面信息的问题
先如下配置
- Exploiting CVE-2015-2509 /MS15-100 : Windows Media Center could allow remote code execution
Exploiting CVE-2015-2509 /MS15-100 : Windows Media Center could allow remote code execution Trend Mi ...
- Eclipse中Copy Qualified Name复制类全名解决办法
Eclipse中用Copy Qualified Name复制类全名时总是这样的/struts1/src/me/edu/HelloAction.java很不方便可以这样解决下载下边插件解压到Eclips ...
- java后端技术
技术概论:Springmvc+mybatis+shiro+Dubbo+ZooKeeper+Redis+KafKa j2ee分布式架构 我在恒生工作,主要开发金融互联网第三方平台的对接项目.目前已经对接 ...
- 图像处理之基础---2个YUV视频 拼接技术
/************************************************* * 主要功能:两路 YUV4:2:0拼接一路左右半宽格式YUV视频 参考资料:http://www ...