hadoop生态系统学习之路（十）MR将结果输出到hbase
之前讲了MR将结果输出到hdfs、hive、db，今天再给大家分享一下，怎样将结果输出到hbase。
首先，提一句，笔者在hadoop集群运行此MR的时候报了一个错误。是一个jar包的缘故，这个错误是hbase版本号的bug，在以下笔者会为大家介绍以及怎样解决问题。
好了，笔者将分以下几个步骤进行介绍：
一、pom依赖
<!-- hbase版本号 -->
<hbase.version>0.96.1.1-cdh5.0.0</hbase.version>
<!-- habase 相关jar-->
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>${hbase.version}</version>
<exclusions>
<exclusion>
<artifactId>jdk.tools</artifactId>
<groupId>jdk.tools</groupId>
</exclusion>
<exclusion>
<artifactId>hbase-protocol</artifactId>
<groupId>org.apache.hbase</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-common</artifactId>
<version>${hbase.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-server</artifactId>
<version>${hbase.version}</version>
<exclusions>
<exclusion>
<artifactId>hbase-protocol</artifactId>
<groupId>org.apache.hbase</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-thrift</artifactId>
<version>${hbase.version}</version>
<exclusions>
<exclusion>
<artifactId>hbase-protocol</artifactId>
<groupId>org.apache.hbase</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-testing-util</artifactId>
<version>${hbase.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
<artifactId>hbase-protocol</artifactId>
<groupId>org.apache.hbase</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-protocol</artifactId>
<version>${hbase.version}</version>
<scope>compile</scope>
</dependency>二、hbase表创建以及hdfs数据准备
首先，我们运行hbase shell进入命令行，然后运行：
create 'user_info', {NAME => 'info'}创建user_info表，列族是info。
然后hdfs数据还是之前入hive以及db的文件，这里就不贴出来了。
三、MR代码实现
首先。我们来看主类：
/**
* Project Name:mr-demo
* File Name:LoadDataToHbaseMR.java
* Package Name:org.qiyongkang.mr.hbasestore
* Date:2016年4月10日下午4:24:11
* Copyright (c) 2016, CANNIKIN(http://http://code.taobao.org/p/cannikin/src/) All Rights Reserved.
*
*/
package org.qiyongkang.mr.hbasestore;
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
/**
* ClassName:LoadDataToHbaseMR <br/>
* Function: TODO ADD FUNCTION. <br/>
* Reason: TODO ADD REASON. <br/>
* Date: 2016年4月10日 下午4:24:11 <br/>
* @author qiyongkang
* @version
* @since JDK 1.6
* @see
*/
public class LoadDataToHbaseMR {
public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
String tablename = "user_info";
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", "172.31.25.8,172.31.25.2,172.31.25.3");
Job job = Job.getInstance(conf, "hbase store");
job.setJarByClass(LoadDataToHbaseMR.class);
job.setMapperClass(HbaseStoreMapper.class);
job.setReducerClass(HbaseStoreReducer.class);
// 设置输入文件路径
FileInputFormat.addInputPath(job, new Path("/qiyongkang/input"));
job.setNumReduceTasks(1);
TableMapReduceUtil.initTableReducerJob(tablename, HbaseStoreReducer.class, job);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(IntWritable.class);
System.exit(job.waitForCompletion(true) ?
0 : 1);
}
}
然后再来看Mapper类：
/**
* Project Name:mr-demo
* File Name:HbaseStoreMapper.java
* Package Name:org.qiyongkang.mr.hbasestore
* Date:2016年4月10日下午4:24:30
* Copyright (c) 2016, CANNIKIN(http://http://code.taobao.org/p/cannikin/src/) All Rights Reserved.
*
*/
package org.qiyongkang.mr.hbasestore;
import java.io.IOException;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;
/**
* ClassName:HbaseStoreMapper <br/>
* Function: TODO ADD FUNCTION. <br/>
* Reason: TODO ADD REASON. <br/>
* Date: 2016年4月10日 下午4:24:30 <br/>
* @author qiyongkang
* @version
* @since JDK 1.6
* @see
*/
public class HbaseStoreMapper extends Mapper<Object, Text, Text, IntWritable>{
private IntWritable one = new IntWritable(1);
@Override
protected void map(Object key, Text value, Mapper<Object, Text, Text, IntWritable>.Context context)
throws IOException, InterruptedException {
context.write(value, one);
}
}
这个Mapper基本上没干啥事。
再来看看Reducer：
/**
* Project Name:mr-demo
* File Name:HbaseStoreReducer.java
* Package Name:org.qiyongkang.mr.hbasestore
* Date:2016年4月10日下午5:01:56
* Copyright (c) 2016, CANNIKIN(http://http://code.taobao.org/p/cannikin/src/) All Rights Reserved.
*
*/
package org.qiyongkang.mr.hbasestore;
import java.io.IOException;
import org.apache.hadoop.hbase.client.Mutation;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.mapreduce.TableReducer;
import org.apache.hadoop.hbase.util.Bytes;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.io.WritableComparable;
import org.apache.hadoop.mapreduce.Reducer;
/**
* ClassName:HbaseStoreReducer <br/>
* Function: TODO ADD FUNCTION. <br/>
* Reason: TODO ADD REASON. <br/>
* Date: 2016年4月10日 下午5:01:56 <br/>
* @author qiyongkang
* @version
* @since JDK 1.6
* @see
*/
public class HbaseStoreReducer extends TableReducer<Text, IntWritable, WritableComparable<Object>>{
@Override
protected void reduce(Text key, Iterable<IntWritable> values,
Reducer<Text, IntWritable, WritableComparable<Object>, Mutation>.Context context)
throws IOException, InterruptedException {
// 每行以制表符分隔 id, account, name, age
String[] strs = key.toString().split("\t");
//id为rowKey
Put put = new Put(strs[0].getBytes());
//account
put.add(Bytes.toBytes("info"), Bytes.toBytes("account"), Bytes.toBytes(strs[1]));
//name
put.add(Bytes.toBytes("info"), Bytes.toBytes("name"), Bytes.toBytes(strs[2]));
//age
put.add(Bytes.toBytes("info"), Bytes.toBytes("age"), Bytes.toBytes(strs[3]));
context.write(null, put);
}
}
我们在reducer再进行split。然后把结果写入到hbase。
四、运行以及版本号bug解决方式
首先，我们打好包。直接在hadoop集群上运行。会报错例如以下：
16/04/10 18:34:25 INFO zookeeper.ClientCnxn: Opening socket connection to server massdata8/172.31.25.8:2181. Will not attempt to authenticate using SASL (unknown error)
java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString我在网上查了半天，结果发现是hbase版本号的原因。
笔者公司hadoop集群所使用的hbase版本号是hbase-0.96.1.1-cdh5.0.0，使用这个刚好遇到这个bug，这个bug在后面的版本号已经解决。
那么，怎样解决呢？大家能够參考此博客：
http://my.oschina.net/cloudcoder/blog/291486
以下，我们运行：
HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf hadoop jar mr-demo-0.0.1-SNAPSHOT-jar-with-dependencies.jar便可运行成功，主要是这个hbase版本号，会动态载入jar文件。后面的日志我们能够看到。例如以下：
HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf hadoop jar mr-demo-0.0.1-SNAPSHOT-jar-with-dependencies.jar
16/04/11 12:54:17 INFO client.RMProxy: Connecting to ResourceManager at massdata8/172.31.25.8:8032
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-cdh5.0.0--1, built on 03/28/2014 04:21 GMT
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:host.name=massdata8
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_75
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/jdk1.7.0_75/jre
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/etc/hadoop/conf:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-beanutils-1.7.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/mockito-all-1.8.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/hue-plugins-3.5.0-cdh5.0.0-SNAPSHOT.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jetty-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jersey-json-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-collections-3.2.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jackson-xc-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-digester-1.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/zookeeper-3.4.5-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/httpclient-4.2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jetty-util-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/avro.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/netty-3.6.2.Final.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jets3t-0.9.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/slf4j-api-1.7.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/jsch-0.1.42.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/httpcore-4.2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/lib/junit-4.8.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-column-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig-bundle-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-cascading.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-encoding-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-thrift-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-avro-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-avro.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-annotations-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-thrift.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-thrift-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-generator.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop-bundle-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-common-2.3.0-cdh5.0.0-tests.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-scrooge.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-encoding-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-common.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-cascading-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-format-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-scrooge-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-encoding.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-common-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-generator-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-column.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-test-hadoop2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-nfs.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-nfs-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-common-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-annotations.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-common-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-cascading-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-auth-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-common.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-format-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig-bundle.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-format.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-avro-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-column-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop-bundle.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//hadoop-auth.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-generator-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-scrooge-sources.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-hadoop-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop/.//parquet-pig-javadoc.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/./:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jetty-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jetty-util-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/netty-3.6.2.Final.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/lib/commons-daemon-1.0.13.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs-2.3.0-cdh5.0.0-tests.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs-nfs.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs-nfs-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-hdfs/.//hadoop-hdfs.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/guice-servlet-3.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jetty-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/javax.inject-1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-json-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/aopalliance-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jackson-xc-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-guice-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/zookeeper-3.4.5-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jetty-util-6.1.26.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jline-0.9.94.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/guice-3.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-client-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-lang-2.6.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/jettison-1.1.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-tests.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-api-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-common-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-client-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-common.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-client.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-common.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-tests-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-api.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/libexec/../../hadoop-yarn/.//hadoop-yarn-common-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/junit-4.10.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jersey-core-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jersey-guice-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/avro.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/commons-io-2.4.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/netty-3.6.2.Final.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/guice-3.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/jersey-server-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib/hamcrest-core-1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-beanutils-1.7.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-extras-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-el-1.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-distcp-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//junit-4.10.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-distcp.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//activation-1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//snappy-java-1.0.4.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-sls-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jsp-api-2.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-datajoin-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jetty-6.1.26.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//guava-11.0.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-rumen-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jersey-json-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-archives.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-streaming.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-collections-3.2.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-math3-3.1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-archives-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//xz-1.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jackson-xc-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-gridmix-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//asm-3.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jersey-core-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-digester-1.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-gridmix.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//zookeeper-3.4.5-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//xmlenc-0.52.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//httpclient-4.2.5.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jetty-util-6.1.26.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-logging-1.1.3.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//metrics-core-3.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//avro.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-io-2.4.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-extras.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jets3t-0.9.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//paranamer-2.3.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//stax-api-1.0-2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//java-xmlbuilder-0.4.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-rumen.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-sls.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//log4j-1.2.17.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-codec-1.4.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jersey-server-1.9.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-lang-2.6.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jettison-1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-auth-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//servlet-api-2.5.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//jsch-0.1.42.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-streaming-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//protobuf-java-2.5.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hamcrest-core-1.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-datajoin.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-auth.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//httpcore-4.2.5.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-net-3.1.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.3.0-cdh5.0.0-tests.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//commons-cli-1.2.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.3.0-cdh5.0.0.jar:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/lib/native
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-279.el6.x86_64
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:user.name=hdfs
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hdfs
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/qiyongkang
16/04/11 12:54:17 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=172.31.25.2:2181,172.31.25.8:2181,172.31.25.3:2181 sessionTimeout=60000 watcher=hconnection-0xfcc6c91, quorum=172.31.25.2:2181,172.31.25.8:2181,172.31.25.3:2181, baseZNode=/hbase
16/04/11 12:54:17 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0xfcc6c91 connecting to ZooKeeper ensemble=172.31.25.2:2181,172.31.25.8:2181,172.31.25.3:2181
16/04/11 12:54:17 INFO zookeeper.ClientCnxn: Opening socket connection to server massdata8/172.31.25.8:2181. Will not attempt to authenticate using SASL (unknown error)
16/04/11 12:54:17 INFO zookeeper.ClientCnxn: Socket connection established to massdata8/172.31.25.8:2181, initiating session
16/04/11 12:54:17 INFO zookeeper.ClientCnxn: Session establishment complete on server massdata8/172.31.25.8:2181, sessionid = 0x353975519d0d25a, negotiated timeout = 60000
16/04/11 12:54:18 INFO mapreduce.TableOutputFormat: Created table instance for user_info
16/04/11 12:54:18 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
16/04/11 12:54:21 INFO input.FileInputFormat: Total input paths to process : 1
16/04/11 12:54:22 INFO mapreduce.JobSubmitter: number of splits:1
16/04/11 12:54:22 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1458262657013_0989
16/04/11 12:54:22 INFO impl.YarnClientImpl: Submitted application application_1458262657013_0989
16/04/11 12:54:22 INFO mapreduce.Job: The url to track the job: http://massdata8:8088/proxy/application_1458262657013_0989/
16/04/11 12:54:22 INFO mapreduce.Job: Running job: job_1458262657013_0989
16/04/11 12:54:32 INFO mapreduce.Job: Job job_1458262657013_0989 running in uber mode : false
16/04/11 12:54:32 INFO mapreduce.Job: map 0% reduce 0%
16/04/11 12:54:42 INFO mapreduce.Job: map 100% reduce 0%
16/04/11 12:54:51 INFO mapreduce.Job: map 100% reduce 100%
16/04/11 12:54:52 INFO mapreduce.Job: Job job_1458262657013_0989 completed successfully
16/04/11 12:54:53 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=182
FILE: Number of bytes written=230209
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=259
HDFS: Number of bytes written=0
HDFS: Number of read operations=2
HDFS: Number of large read operations=0
HDFS: Number of write operations=0
Job Counters
Launched map tasks=1
Launched reduce tasks=1
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=7763
Total time spent by all reduces in occupied slots (ms)=8240
Total time spent by all map tasks (ms)=7763
Total time spent by all reduce tasks (ms)=8240
Total vcore-seconds taken by all map tasks=7763
Total vcore-seconds taken by all reduce tasks=8240
Total megabyte-seconds taken by all map tasks=7949312
Total megabyte-seconds taken by all reduce tasks=8437760
Map-Reduce Framework
Map input records=8
Map output records=8
Map output bytes=168
Map output materialized bytes=178
Input split bytes=117
Combine input records=0
Combine output records=0
Reduce input groups=8
Reduce shuffle bytes=178
Reduce input records=8
Reduce output records=8
Spilled Records=16
Shuffled Maps =1
Failed Shuffles=0
Merged Map outputs=1
GC time elapsed (ms)=70
CPU time spent (ms)=2560
Physical memory (bytes) snapshot=960487424
Virtual memory (bytes) snapshot=3228942336
Total committed heap usage (bytes)=1584398336
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=142
File Output Format Counters
Bytes Written=0这样，便运行成功啦。
五、查看结果
以下，我们在hbase运行scan ‘user_info’，能够看到：
说明结果入hbase成功！
ok，就说到这儿了。事实上也能够读取hbase的表数据，大家也能够去研究一下。
hadoop生态系统学习之路（十）MR将结果输出到hbase的更多相关文章
- hadoop生态系统学习之路（六）hive的简单使用
一.hive的基本概念与原理 Hive是基于Hadoop之上的数据仓库,能够存储.查询和分析存储在 Hadoop 中的大规模数据. Hive 定义了简单的类 SQL 查询语言,称为 HQL.它同意熟悉 ...
- hadoop生态系统学习之路（八）hbase与hive的数据同步以及hive与impala的数据同步
在之前的博文中提到,hive的表数据是能够同步到impala中去的. 一般impala是提供实时查询操作的,像比較耗时的入库操作我们能够使用hive.然后再将数据同步到impala中.另外,我们也能够 ...
- Hadoop生态系统学习路线
主要介绍Hadoop家族产品,经常使用的项目包含Hadoop, Hive, Pig, HBase, Sqoop, Mahout, Zookeeper, Avro, Ambari, Chukwa.新添加 ...
- zigbee学习之路(十五):基于协议栈的按键实验
一.前言 经过上次的学习,相信大家已经初步学会使用zigbee协议进行发送和接受数据了.今天,我们要进行的实验是按键的实验,学会如何在协议栈里实现按键中断. 二.实验功能 在协议栈上实现按键中断,BU ...
- zigbee学习之路(十二):zigbee协议原理介绍
一.前言 从今天开始,我们要正式开始进行zigbee相关的通信实验了,我所使用的协议栈是ZStack 是TI ZStack-CC2530-2.3.0-1.4.0版本,大家也可以从TI的官网上直接下载T ...
- zigbee学习之路(十):串口(接收)
一.前言 上次我们学习了串口的发送,今天我们要学习串口的接收,要实现的功能是接收电脑发来的数据,控制LED 灯闪烁,而且将收到的数据发回给电脑显示出来.而且要采用不占用cpu资源的中断. 二原理与分析 ...
- 学习之路十四：客户端调用WCF服务的几种方法小议
最近项目中接触了一点WCF的知识,也就是怎么调用WCF服务,上网查了一些资料,很快就搞出来,可是不符合头的要求,主要有以下几个方面: ①WCF的地址会变动,地址虽变,但是里面的逻辑不变! ②不要引用W ...
- Object-c学习之路十二（OC的copy）
oc中的拷贝分为:copy(浅拷贝)和mutablecopy(深拷贝). 浅拷贝也为指针拷贝,拷贝后原来的对象计数器会+1: 深拷贝为对象拷贝,原来的对象计数器不变. 注意:自定义对象拷贝时要实现NS ...
- 嵌入式Linux驱动学习之路(十九)触摸屏驱动、tslib测试
触摸屏使用流程: 1. 按下产生中断. 2.在中断处理程序中启动AD转换XY坐标. 3.AD转换结束并产生AD中断. 4. 在AD的中断处理函数中上报信息,启动定时器. 5. 定时器时间到后进入中断, ...
随机推荐
- 【BZOJ 1222】 [HNOI2001] 产品加工（DP）
Description 某加工厂有A.B两台机器,来加工的产品可以由其中任何一台机器完成,或者两台机器共同完成.由于受到机器性能和产品特性的限制,不同的机器加工同一产品所需的时间会不同,若同时由两台机 ...
- 【02】sass更新的方法
[02]更新的方法 gem install sass **
- Couchbase V(管理任务)
Couchbase V(管理任务) 多读写 在Couchbase2.1中支持硬盘多读些(Multi- Readers and Writers),一般双核4G服务默认3个thread 4核16G内存一个 ...
- 大数据学习——mapreduce运营商日志增强
需求 1.对原始json数据进行解析,变成普通文本数据 2.求出每个人评分最高的3部电影 3.求出被评分次数最多的3部电影 数据 https://pan.baidu.com/s/1gPsQXVYSQE ...
- Linux使用Mutt发送邮件/附件
使用Mutt发邮件极其方便,只需要一条命令即可发送或者批量发送邮件 功能说明:E-mail管理程序. 语 法:mutt [-hnpRvxz][-a<文件>][-b<地址>][- ...
- MyBatis使用Mapper动态代理开发Dao层
开发规范 Mapper接口开发方法只需要程序员编写Mapper接口(相当于Dao接口),由Mybatis框架根据接口定义创建接口的动态代理对象,代理对象的方法体同原始Dao接口实现类方法. Mappe ...
- [luoguP3606] [USACO17JAN]Building a Tall Barn建谷仓（贪心 + 线段树）
传送门 把线段都读进来然后排序,先按右端点为第一关键字从小到大排序,后按左端点为第二关键字从小到大排序. 注意不能先按左端点后按右端点排序,否则会出现大包小的情况,如下: —————— ——— — ...
- 实体类与实体DTO类之间的转换
实体类与实体DTO类之间的转换 实体类与实体DTO类之间的转换 1.通过使用第三方序列化反序列化工具Newtonsoft.Json 2.通过反射实现 3.通过表达式目录树加字典缓存实现 4. 通过表达 ...
- msp430项目编程01
msp430中项目---点阵LED显示 1.点阵LED介绍 2.代码(直接使用引脚驱动) 3.代码(使用芯片驱动) 4.项目总结 msp430项目编程 msp430入门学习
- poj2773求第K个与m互质的数
//半年前做的,如今回顾一下,还是有所收货的,数的唯一分解,.简单题. #include<iostream> #include<cstring> using namespace ...