hadoop 2.6配置记录
本地hadoop配置
1)core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/tmp</value>
<description>A base for other temporary directories</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://nameNode:10000</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>4096</value>
</property>
</configuration>
2)hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>dfs.nameservices</name>
<value>hadoop-xxxxx</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>nameNode:50090</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///home/hadoop/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///home/hadoop/dfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>
3) mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobtracker.http.address</name>
<value>nameNode:50030</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>nameNode:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>nameNode:19888</value>
</property>
</configuration>
4) yarn-site.xml
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration> <!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>nameNode:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>nameNode:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>nameNode:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>nameNode:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>nameNode:8088</value>
</property>
</configuration>
5)slaves
dataNode
6) hadoop-env.sh和yarn-env.sh
... export JAVA_HOME=/path/to/jdk ...
7) 配置完执行
# hadoop namenode -format
记得配置PATH环境变量
8)查看状态
http://ejiajie-thinkpad-t450s:50070
http://ejiajie-thinkpad-t450s:8088
环境:ubuntu
hadoop 2.6配置记录的更多相关文章
- Hadoop 集群配置记录小结
Hadoop集群配置往往按照网上教程就可以"配置成功",但是你自己在操作的时候会有很多奇奇怪怪的问题出现, 我在这里整理了一下常见的问题与处理方法: 1.配置/etc/hosts ...
- Hadoop安装测试简单记录
安装的节点如下:1个namenode.1个hiveserver.3个dataNode192.168.1.139 namenode1192.168.1.146 hiveserver 192.16 ...
- UBuntu安装配置记录
记得是06年左右第一次安装的 Linux,当时是下载的 Fedora镜像,版本已经记不清了,在商业街的电脑维修店刻的盘,回来后兴冲冲地和XP一起安装的双系统.其实就是直接的体验了一把,只是看了看X-W ...
- 五十九.大数据、Hadoop 、 Hadoop安装与配置 、 HDFS
1.安装Hadoop 单机模式安装Hadoop 安装JAVA环境 设置环境变量,启动运行 1.1 环境准备 1)配置主机名为nn01,ip为192.168.1.21,配置yum源(系统源) 备 ...
- 初识Hadoop一,配置及启动服务
一.Hadoop简介: Hadoop是由Apache基金会所开发的分布式系统基础架构,实现了一个分布式文件系统(Hadoop Distributed File System),简称HDFS:Hadoo ...
- [Hadoop入门] - 2 ubuntu安装与配置 hadoop安装与配置
ubuntu安装(这里我就不一一捉图了,只引用一个网址, 相信大家能力) ubuntu安装参考教程: http://jingyan.baidu.com/article/14bd256e0ca52eb ...
- Hadoop伪分布模式配置
本作品由Man_华创作,采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可.基于http://www.cnblogs.com/manhua/上的作品创作. 请先按照上一篇文章H ...
- Hadoop伪分布配置与基于Eclipse开发环境搭建
国内私募机构九鼎控股打造APP,来就送 20元现金领取地址:http://jdb.jiudingcapital.com/phone.html内部邀请码:C8E245J (不写邀请码,没有现金送)国内私 ...
- Hadoop集群配置(最全面总结)
Hadoop集群配置(最全面总结) 通常,集群里的一台机器被指定为 NameNode,另一台不同的机器被指定为JobTracker.这些机器是masters.余下的机器即作为DataNode也作为Ta ...
随机推荐
- mysql查看修改字符集
1.列出MYSQL支持的所有字符集: SHOW CHARACTER SET; 2.当前MYSQL服务器字符集设置 SHOW VARIABLES LIKE 'character_set_%'; 3.当前 ...
- Concurrency vs. Parallelism
http://getakka.net/docs/concepts/terminology Terminology and Concepts In this chapter we attempt to ...
- Webpack参考资料
学习是一种进步,只有不断的向别人学习,才能提升自己.三人行必有我师焉 1. http://www.cnblogs.com/zhengjialux/p/5861845.html
- SQL Server优化常用SQL语句
--所有没有主键的表 select name from sysobjects where xtype='U' and id not in ( select i.parent_obj from syso ...
- Codeforces 722D. Generating Sets
D. Generating Sets time limit per test 2 seconds memory limit per test 256 megabytes input standard ...
- Hibernate学习笔记(一)
2016/4/18 19:58:58 Hibernate学习笔记(一) 1.Hibernate框架的概述: 就是一个持久层的ORM框架. ORM:对象关系映射.将Java中实体对象与关系型数据库中表建 ...
- 如何监控checkbox改变
现在想实现一个效果,即使弹幕开关,弹幕开关实际是checkbox $(document).on('change',"#barrage",function(){ if (!$(thi ...
- java 执行jar指定log4j.properties文件位置
默认情况下,log4j.properties会被加载,并且这个文件需要在classpath根目录,当打包jar时,会打包打jar内部,当需要修改日志级别时,会比较麻烦 可以使用:java -jar - ...
- 2017 New Year’s Greetings from Sun Yat-sen University
As winter turns to spring, the world around us begins to take on an air of freshness. As 2017 is fa ...
- ubuntu14.04 Hadoop单机开发环境搭建MapReduce项目
Hadoop官网:http://hadoop.apache.org/ 目前最新的版本是Hadoop 3.0.0-alpha1前提:java 1.6 版本以上 首先从官网下载压缩包(hadoop-3.0 ...