Hive报错之java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory

一、问题：

在使用Hive0.11进行select查询的时候报：

hive> create table zhifu2 as select distinct uid,substr(time,1,12),site from zhifu;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201403211432_0001, Tracking URL = http://node1:60030/jobdetails.jsp?jobid=job_201403211432_0001
Kill Command = /app/hadoop/bin/../bin/hadoop job  -kill job_201403211432_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2014-03-21 15:51:13,341 Stage-1 map = 0%,  reduce = 0%
2014-03-21 15:52:14,121 Stage-1 map = 0%,  reduce = 0%
2014-03-21 15:53:15,019 Stage-1 map = 0%,  reduce = 0%
2014-03-21 15:54:21,120 Stage-1 map = 0%,  reduce = 0%
2014-03-21 15:55:55,130 Stage-1 map = 0%,  reduce = 0%
2014-03-21 15:57:40,915 Stage-1 map = 0%,  reduce = 0%
2014-03-21 15:59:28,636 Stage-1 map = 0%,  reduce = 0%
Ended Job = job_201403211432_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://node1:60030/jobdetails.jsp?jobid=job_201403211432_0001
Examining task ID: task_201403211432_0001_m_000002 (and more) from job job_201403211432_0001
null
Exception in thread "Thread-46" java.lang.NullPointerException
    at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:313)
    at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:118)
    at java.lang.Thread.run(Thread.java:722)
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

查看jobtracker页面中该job的信息发现有：

java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 5 more Caused by: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) ... 10 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 13 more Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121) ... 18 more Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple.(GenericUDTFJSONTuple.java:56) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:526) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:520) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.(FunctionRegistry.java:423) at org.apache.hadoop.hive.ql.exec.DefaultUDFMethodResolver.getEvalMethod(DefaultUDFMethodResolver.java:59) at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:154) at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:111) at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:141) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:227) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:543) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:100) ... 18 more Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.JsonFactory at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) ... 44 more

其中Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory是引起问题的原因。
二、原因：

Hadoop版本是0.20.2的，在$HADOOP_HOME/lib中不包含：

[hadoop@node2 lib]$ ll jackson*
-rw-rw-r-- 1 hadoop hadoop 227500 2012-01-21 09:24 jackson-core-asl-1.8.8.jar
-rw-rw-r-- 1 hadoop hadoop  17884 2012-01-21 09:26 jackson-jaxrs-1.8.8.jar
-rw-rw-r-- 1 hadoop hadoop 668564 2012-01-21 09:24 jackson-mapper-asl-1.8.8.jar
-rw-rw-r-- 1 hadoop hadoop  32353 2012-01-21 09:26 jackson-xc-1.8.8.jar

这四个包，因此会报错。

三、解决方案：

方法有两种：

1、在集群各节点上的hadoop的lib包增加相关的4个jar包，并重启集群。

2、hive prompt里输入：

hive> add jars /app/hive/lib/jackson-core-asl-1.8.8.jar jackson-jaxrs-1.8.8.jar jackson-mapper-asl-1.8.8.jar jackson-xc-1.8.8.jar
    > ;
Added /app/hive/lib/jackson-core-asl-1.8.8.jar to class path
Added resource: /app/hive/lib/jackson-core-asl-1.8.8.jar
Added jackson-jaxrs-1.8.8.jar to class path
Added resource: jackson-jaxrs-1.8.8.jar
Added jackson-mapper-asl-1.8.8.jar to class path
Added resource: jackson-mapper-asl-1.8.8.jar
Added jackson-xc-1.8.8.jar to class path
Added resource: jackson-xc-1.8.8.jar

然后再执行select操作