Hadoop官网提供的编译好的hadoop-2.3.0.tar.gz二进制包是在32位系统上编译的,在64系统上运行会有一些错误,比如:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

此时需要自行编译hadoop 2.30 源码。本人编译的hadoop 2.30的二进制包经实验证明可以正确安装并正确运行了Word Count程序。此处可以下载我编译的二进制包:

http://pan.baidu.com/s/1eQrgsWa

以下为编译步骤

1. 更新软件包列表

apt-get update

2. 安装编译所需要的软件: 为什么需要这些依赖包? 我也不知道==

apt-get install -y openjdk-7-jdk libprotobuf-dev protobuf-compiler maven cmake build-essential pkg-config libssl-dev zlib1g-dev llvm-gcc automake autoconf make

3. 下载hadoop 2.30的源文件包

wget http://archive.apache.org/dist/hadoop/core/hadoop-2.3.0/hadoop-2.3.0-src.tar.gz

4. 解压hadoop 2.30 的源文件包

tar -xzvf hadoop-2.3.0-src.tar.gz

5. 进入hadoop 2.30 文件夹

cd hadoop-2.3.0-src

6. 编译hadoop 2.30 源文件

mvn package -Pdist,native -DskipTests –Dtar

正确执行的结果如下:

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO]

[INFO] Apache Hadoop Main ................................ SUCCESS [1:11.968s]

[INFO] Apache Hadoop Project POM ......................... SUCCESS [30.393s]

[INFO] Apache Hadoop Annotations ......................... SUCCESS [18.398s]

[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.246s]

[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [20.372s]

[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [23.721s]

[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [1:41.836s]

[INFO] Apache Hadoop Auth ................................ SUCCESS [22.303s]

[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [7.052s]

[INFO] Apache Hadoop Common .............................. SUCCESS [2:29.466s]

[INFO] Apache Hadoop NFS ................................. SUCCESS [11.604s]

[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.073s]

[INFO] Apache Hadoop HDFS ................................ SUCCESS [1:30.230s]

[INFO] Apache Hadoop HttpFS .............................. SUCCESS [17.976s]

[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [19.927s]

[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [3.304s]

[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.032s]

[INFO] hadoop-yarn ....................................... SUCCESS [0.033s]

[INFO] hadoop-yarn-api ................................... SUCCESS [36.284s]

[INFO] hadoop-yarn-common ................................ SUCCESS [33.912s]

[INFO] hadoop-yarn-server ................................ SUCCESS [0.213s]

[INFO] hadoop-yarn-server-common ......................... SUCCESS [8.193s]

[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [41.181s]

[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [2.768s]

[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [13.923s]

[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.904s]

[INFO] hadoop-yarn-client ................................ SUCCESS [4.363s]

[INFO] hadoop-yarn-applications .......................... SUCCESS [0.120s]

[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [2.262s]

[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [1.615s]

[INFO] hadoop-yarn-site .................................. SUCCESS [0.086s]

[INFO] hadoop-yarn-project ............................... SUCCESS [2.703s]

[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.132s]

[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [18.951s]

[INFO] hadoop-mapreduce-client-common .................... SUCCESS [14.320s]

[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [3.330s]

[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [9.664s]

[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [7.678s]

[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [9.263s]

[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [1.549s]

[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [5.748s]

[INFO] hadoop-mapreduce .................................. SUCCESS [2.880s]

[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [7.080s]

[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [14.648s]

[INFO] Apache Hadoop Archives ............................ SUCCESS [2.602s]

[INFO] Apache Hadoop Rumen ............................... SUCCESS [5.706s]

[INFO] Apache Hadoop Gridmix ............................. SUCCESS [3.649s]

[INFO] Apache Hadoop Data Join ........................... SUCCESS [2.483s]

[INFO] Apache Hadoop Extras .............................. SUCCESS [2.678s]

[INFO] Apache Hadoop Pipes ............................... SUCCESS [6.359s]

[INFO] Apache Hadoop OpenStack support ................... SUCCESS [5.088s]

[INFO] Apache Hadoop Client .............................. SUCCESS [4.534s]

[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.433s]

[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [7.757s]

[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [4.099s]

[INFO] Apache Hadoop Tools ............................... SUCCESS [0.428s]

[INFO] Apache Hadoop Distribution ........................ SUCCESS [18.045s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 14:59.240s

[INFO] Finished at: Thu Jan 15 18:51:59 JST 2015

[INFO] Final Memory: 168M/435M

[INFO] ------------------------------------------------------------------------

编译好的二进制文件包位于

hadoop-2.3.0-src/hadoop-dist/target/hadoop-2.3.0.tar.gz

PS: 使用自行编译的hadoop 2.30二进制包安装hadoop 2.30时需要注意删除 .bashrc文件与hadoop-env.sh文件中下面两行(默认不会有这两行,但是尝试解决报错时可能改写了)

export HADOOP_COMMON_LIB_NATIVE_DIR="~/hadoop/lib/"

export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=~/hadoop/lib/"

linux编译64bitHadoop (eg: ubuntu14.04 and hadoop 2.3.0)的更多相关文章

  1. Torch,Tensorflow使用: Ubuntu14.04(x64)+ CUDA8.0 安装 Torch和Tensorflow

    系统配置: Ubuntu14.04(x64) CUDA8.0 cudnn-8.0-linux-x64-v5.1.tgz(Tensorflow依赖) Anaconda 1. Torch安装 Torch是 ...

  2. 二、Ubuntu14.04下安装Hadoop2.4.0 (伪分布模式)

    在Ubuntu14.04下安装Hadoop2.4.0 (单机模式)基础上配置 一.配置core-site.xml /usr/local/hadoop/etc/hadoop/core-site.xml ...

  3. Ubuntu 12.04下Hadoop 2.2.0 集群搭建(原创)

    现在大家可以跟我一起来实现Ubuntu 12.04下Hadoop 2.2.0 集群搭建,在这里我使用了两台服务器,一台作为master即namenode主机,另一台作为slave即datanode主机 ...

  4. 搭建ubuntu14.04的hadoop集群【docker容器充当服务器】

    首先弄出来装有hadoop.java.ssh.vim的镜像起名badboyf/hadoop.做镜像有两种方法,一种是用Dockerfile来生成一个镜像,一种是基于ubuntu14.04的基础镜像生成 ...

  5. Ubuntu14.04 安装配置Hadoop2.6.0

    目前关于Hadoop的安装配置教程书上.官方教程.博客都有很多,但由于对Linux环境的不熟悉以及各种教程或多或少有这样那样的坑,很容易导致折腾许久都安装不成功(本人就是受害人之一).经过几天不断尝试 ...

  6. Ubuntu14.04安装和配置Tomcat8.0.12(转)

    Ubuntu14.04长的好看,所以一时间很感兴趣,研究各种软件的安装和开发环境的配置.今天先把安装的tomcat 8.0.12的教程分享给大家.如果你需要,请收藏!!!   工具/原料 系统环境:U ...

  7. Ubuntu14.04安装和配置Tomcat8.0.12

    Ubuntu14.04长的好看,所以一时间很感兴趣,研究各种软件的安装和开发环境的配置.今天先把安装的tomcat 8.0.12的教程分享给大家.如果你需要,请收藏!!!   官方网站下载最新的tom ...

  8. Linux系统安装及初始化(ubuntu14.04)

    Windows 7下硬盘安装Ubuntu 14.04图文教程 Ubuntu 官方已经发布了正式版的 Ubuntu 14.04 LTS,并宣称这是为云计算准备的版本.该版本在云平台和伸缩环境的可靠性.性 ...

  9. ubuntu14.04下配置使用openCV3.0

    [操  作  系  统] Ubuntu 14.04 LTS [OpenCV版本]  3.0.0-beta [Eclipse 版 本] 3.8.1 需要知识: Linux系统shell命令基础 编译原理 ...

随机推荐

  1. sql server查询语句条件判断字段值是否为NULL

    判断字段是否为null select * from table where c is null    select * from table where c is not null 判断字段是否为空 ...

  2. c# 后台分页 jqgrid

    /// <summary> /// 获取设备数量 /// </summary> /// <param name="Organid">单位ID&l ...

  3. 31.QPainter-rotate()函数分析-文字旋转不倾斜,图片旋转实现等待

    在上章和上上上章: 28.QT-QPainter介绍 30.QT-渐变之QLinearGradient. QConicalGradient.QRadialGradient 学习了QPainter基础绘 ...

  4. 一个3年Java程序员的坎与选择

    前言 LZ 15年本科毕业,不知不觉3年过去了,去年底裸辞回到成都来发展,年后开始找工作,面试了几家公司,现在整理整理做个总结,也方便规划下一个3到5年以及和广大想要进阶的Java程序员同胞们共勉. ...

  5. 12. ReadWriteLock 读写锁

    package com.gf.demo11; import java.util.concurrent.locks.ReadWriteLock; import java.util.concurrent. ...

  6. Get与Post的主要区别

    这里附一篇自己的简短理解 get相对于post更不安全,虽然都可以加密 get的参数会显示在浏览器地址栏中,而post的参数不会显示在浏览器地址栏中: 使用post提交的页面在点击[刷新]按钮的时候浏 ...

  7. MongoDB复合索引详解

    摘要: 对于MongoDB的多键查询,创建复合索引可以有效提高性能. 什么是复合索引? 复合索引,即Compound Index,指的是将多个键组合到一起创建索引,这样可以加速匹配多个键的查询.不妨通 ...

  8. js之单例模式

    单例模式是指一个类,只有一个实例.实现的思路是,创建实例时候加判断,如果有实例则返回,如果没有就new一个,并返回. 第一步: 创建类. function Waiter(id, name, salar ...

  9. setTimeout()与clearTimeout()

    setTimeout(code,millisec)setTimeout() 只执行 code 一次.如果要多次调用,请使用 setInterval() 或者让 code 自身再次调用 setTimeo ...

  10. jQ效果:jQuery之插件开发短信发送倒计时功能

    实现的主要功能如下: 1.点击按钮的时候,可以进行倒计时,倒计时自定义. 2.当接收短信失败后,倒计时停止,可点击重新发送短信. 3.点击的元素支持一般标签和input标签. html代码: < ...