hadoop 2.6配置记录
本地hadoop配置
1)core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/tmp</value>
<description>A base for other temporary directories</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://nameNode:10000</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>4096</value>
</property>
</configuration>
2)hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>dfs.nameservices</name>
<value>hadoop-xxxxx</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>nameNode:50090</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///home/hadoop/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///home/hadoop/dfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>
3) mapred-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobtracker.http.address</name>
<value>nameNode:50030</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>nameNode:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>nameNode:19888</value>
</property>
</configuration>
4) yarn-site.xml
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration> <!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>nameNode:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>nameNode:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>nameNode:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>nameNode:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>nameNode:8088</value>
</property>
</configuration>
5)slaves
dataNode
6) hadoop-env.sh和yarn-env.sh
... export JAVA_HOME=/path/to/jdk ...
7) 配置完执行
# hadoop namenode -format
记得配置PATH环境变量
8)查看状态
http://ejiajie-thinkpad-t450s:50070
http://ejiajie-thinkpad-t450s:8088
环境:ubuntu
hadoop 2.6配置记录的更多相关文章
- Hadoop 集群配置记录小结
Hadoop集群配置往往按照网上教程就可以"配置成功",但是你自己在操作的时候会有很多奇奇怪怪的问题出现, 我在这里整理了一下常见的问题与处理方法: 1.配置/etc/hosts ...
- Hadoop安装测试简单记录
安装的节点如下:1个namenode.1个hiveserver.3个dataNode192.168.1.139 namenode1192.168.1.146 hiveserver 192.16 ...
- UBuntu安装配置记录
记得是06年左右第一次安装的 Linux,当时是下载的 Fedora镜像,版本已经记不清了,在商业街的电脑维修店刻的盘,回来后兴冲冲地和XP一起安装的双系统.其实就是直接的体验了一把,只是看了看X-W ...
- 五十九.大数据、Hadoop 、 Hadoop安装与配置 、 HDFS
1.安装Hadoop 单机模式安装Hadoop 安装JAVA环境 设置环境变量,启动运行 1.1 环境准备 1)配置主机名为nn01,ip为192.168.1.21,配置yum源(系统源) 备 ...
- 初识Hadoop一,配置及启动服务
一.Hadoop简介: Hadoop是由Apache基金会所开发的分布式系统基础架构,实现了一个分布式文件系统(Hadoop Distributed File System),简称HDFS:Hadoo ...
- [Hadoop入门] - 2 ubuntu安装与配置 hadoop安装与配置
ubuntu安装(这里我就不一一捉图了,只引用一个网址, 相信大家能力) ubuntu安装参考教程: http://jingyan.baidu.com/article/14bd256e0ca52eb ...
- Hadoop伪分布模式配置
本作品由Man_华创作,采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可.基于http://www.cnblogs.com/manhua/上的作品创作. 请先按照上一篇文章H ...
- Hadoop伪分布配置与基于Eclipse开发环境搭建
国内私募机构九鼎控股打造APP,来就送 20元现金领取地址:http://jdb.jiudingcapital.com/phone.html内部邀请码:C8E245J (不写邀请码,没有现金送)国内私 ...
- Hadoop集群配置(最全面总结)
Hadoop集群配置(最全面总结) 通常,集群里的一台机器被指定为 NameNode,另一台不同的机器被指定为JobTracker.这些机器是masters.余下的机器即作为DataNode也作为Ta ...
随机推荐
- 微信小程序组件-----城市切换
直接上地址,有需要的直接下载,使用: https://github.com/chenjinxinlove/citySelect
- Android混淆打包
一.理论知识 ProGuard是一款免费的Java类文件压缩器.优化器和混淆器.它能发现并删除无用类.字段(field).方法和属性值(attribute).它也能优化字节码并删除无用的指令.最后 ...
- UIButton的文本与图片的布局
UIButton内部文本和图片的布局是我们日常代码中,不可缺少的部分,按钮默认左边图片右边文本,那要实现左边文本,右边图片,我们该怎么解决呢,上面图片,下面文本又该怎么办呢 其实很简单,今天总结下,目 ...
- OpenCV从入门到放弃系列之——core模块.核心功能(一)
Mat - 基本图像容器 世间的图像是各种各样的,但是到了计算机的世界里所有的图像都简化为了数值矩以及矩阵信息.作为一个计算视觉库,OpenCV的主要目的就是处理和操作这些信息,来获取更高级的信息,也 ...
- C# 只移除最后一个字符
string str = "|||"; Console.WriteLine(str.Substring(0, str.Length - 1)); Console.WriteLine ...
- php : MVC 演示(使用单例工厂)
此例子是MVC的简单应用, 要达到的效果如下: 用户列表: 姓名 年龄 学历 兴趣 出生地 账号创建时间 操作 keen 20 高中 篮球,足球 广东 2016-11-08 10:00:31 删除 a ...
- 用:hover伪类代替js的hover事件
制作二级菜单要实现鼠标移动上去显示子菜单,鼠标移出子菜单隐藏,或者其他类似需求的地方,首先我会想到用jquery的hover事件来实现,如: $(".nav").hover(fun ...
- ajax请求总是进入Error里
ajax请求时找到了正确的地址,执行完返回总是进入error里,并且浏览器错误显示是找不到请求的地址. 解决办法: 查看配置文件的,把maxJsonLength值改大. <system.web. ...
- CSS垂直居中的方法
前端开发过程中,水平垂直居中是比较常用的.下面直接开门见山,看看不同方法实现垂直居中的各自优点和其不足之处. 1.将“line-height”和“height”设置成一致 这种方法用来实现单行垂直居中 ...
- Node.js 基础库
全局对象 Node.js 中的全局对象是 global,所有全局变量(除了 global 本身以外)都是 global对象的属性. 我们在 Node.js 中能够直接访问到对象通常都是 global ...