本地hadoop配置

1)core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/tmp</value>
<description>A base for other temporary directories</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://nameNode:10000</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>4096</value>
</property>
</configuration>

2)hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>dfs.nameservices</name>
<value>hadoop-xxxxx</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>nameNode:50090</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///home/hadoop/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///home/hadoop/dfs/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
</configuration>

3) mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
--> <!-- Put site-specific property overrides in this file. --> <configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobtracker.http.address</name>
<value>nameNode:50030</value>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>nameNode:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>nameNode:19888</value>
</property>
</configuration>

4) yarn-site.xml

<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration> <!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>nameNode:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>nameNode:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>nameNode:8031</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>nameNode:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>nameNode:8088</value>
</property>
</configuration>

5)slaves

dataNode

6) hadoop-env.sh和yarn-env.sh

...

export JAVA_HOME=/path/to/jdk

...

7) 配置完执行

# hadoop namenode -format

记得配置PATH环境变量

8)查看状态

http://ejiajie-thinkpad-t450s:50070

http://ejiajie-thinkpad-t450s:8088

环境:ubuntu

hadoop 2.6配置记录的更多相关文章

  1. Hadoop 集群配置记录小结

    Hadoop集群配置往往按照网上教程就可以"配置成功",但是你自己在操作的时候会有很多奇奇怪怪的问题出现, 我在这里整理了一下常见的问题与处理方法: 1.配置/etc/hosts ...

  2. Hadoop安装测试简单记录

    安装的节点如下:1个namenode.1个hiveserver.3个dataNode192.168.1.139   namenode1192.168.1.146   hiveserver 192.16 ...

  3. UBuntu安装配置记录

    记得是06年左右第一次安装的 Linux,当时是下载的 Fedora镜像,版本已经记不清了,在商业街的电脑维修店刻的盘,回来后兴冲冲地和XP一起安装的双系统.其实就是直接的体验了一把,只是看了看X-W ...

  4. 五十九.大数据、Hadoop 、 Hadoop安装与配置 、 HDFS

    1.安装Hadoop 单机模式安装Hadoop 安装JAVA环境 设置环境变量,启动运行   1.1 环境准备   1)配置主机名为nn01,ip为192.168.1.21,配置yum源(系统源) 备 ...

  5. 初识Hadoop一,配置及启动服务

    一.Hadoop简介: Hadoop是由Apache基金会所开发的分布式系统基础架构,实现了一个分布式文件系统(Hadoop Distributed File System),简称HDFS:Hadoo ...

  6. [Hadoop入门] - 2 ubuntu安装与配置 hadoop安装与配置

    ubuntu安装(这里我就不一一捉图了,只引用一个网址, 相信大家能力) ubuntu安装参考教程:  http://jingyan.baidu.com/article/14bd256e0ca52eb ...

  7. Hadoop伪分布模式配置

    本作品由Man_华创作,采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可.基于http://www.cnblogs.com/manhua/上的作品创作. 请先按照上一篇文章H ...

  8. Hadoop伪分布配置与基于Eclipse开发环境搭建

    国内私募机构九鼎控股打造APP,来就送 20元现金领取地址:http://jdb.jiudingcapital.com/phone.html内部邀请码:C8E245J (不写邀请码,没有现金送)国内私 ...

  9. Hadoop集群配置(最全面总结)

    Hadoop集群配置(最全面总结) 通常,集群里的一台机器被指定为 NameNode,另一台不同的机器被指定为JobTracker.这些机器是masters.余下的机器即作为DataNode也作为Ta ...

随机推荐

  1. 在AngularJS中同一个页面配置一个或者多个ng-app

    在AngularJS学习中,对于ng-app初始化一个AngularJS程序属性的使用需要注意,在一个页面中AngularJS自动加载第一个ng-app,其他ng-app会忽略, 如果需要加载其他ng ...

  2. R语言解读多元线性回归模型

    转载:http://blog.fens.me/r-multi-linear-regression/ 前言 本文接上一篇R语言解读一元线性回归模型.在许多生活和工作的实际问题中,影响因变量的因素可能不止 ...

  3. MATLAB地图工具箱学习总结(一)从地图投影说起

    MATLAB地图工具箱学习总结(一)从地图投影说起 前言 本学期地图投影课上,李连营老师建议我们使用MATLAB完成每周的作业.从大二上学期开始接触MATLAB学习数学运算和地理数据处理的我,自然不会 ...

  4. FPGA重要设计思想

    FPGA重要设计思想   1.速度和面积互换原则.以面积换速度可以实现很高的数据吞吐率,其实串/并转换.就是一种以面积换速度的思想 2.乒乓操作. 3.串/并转换的思想. 高速数据处理的重要技巧之一. ...

  5. CentOS 7 最小化安装的无线网络配置

    1.首先下载iw工具. yum -y install iw 2.获取无线网卡的名称 执行iw dev,假设获得名称为 wlp3s0(示例) 3.激活无线网络接口 执行ip link set wlp3s ...

  6. My first python script for work

    I write it yesterday to watch the NE process(rcpfd,cfgd) automatically, then i will write a window t ...

  7. Node聊天程序实例02:chat_server.js

    作者:vousiu 出处:http://www.cnblogs.com/vousiu 本实例参考自Mike Cantelon等人的<Node.js in Action>一书. chat_s ...

  8. jQuery 查找父元素

    function deletesec1Div5(obj){ $(obj).closest(".sec1-div5").remove();}自己写的一段代码,实现了table中的全选 ...

  9. win7 32位安装php redis驱动

    根据php版本下载php_redis扩展,然后把php_redis.dll这个扩展文件放到php中的ext文件夹中,然后再去php-ini中添加一行代码:extension=php_redis.dll ...

  10. 有关宏定义的bug

    #define max(a,b) a>b?a:b 这个宏有诸多缺点:当调用比如max(3,2)*2时会出现不想要的结果.按理来说,我们想要得到的结果是:3*2=6,但是由于优先级的问题,得到的结 ...