一、问题:

在使用Hive0.11进行select查询的时候报:

hive> create table zhifu2 as select distinct uid,substr(time,1,12),site from zhifu;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201403211432_0001, Tracking URL = http://node1:60030/jobdetails.jsp?jobid=job_201403211432_0001
Kill Command = /app/hadoop/bin/../bin/hadoop job -kill job_201403211432_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2014-03-21 15:51:13,341 Stage-1 map = 0%, reduce = 0%
2014-03-21 15:52:14,121 Stage-1 map = 0%, reduce = 0%
2014-03-21 15:53:15,019 Stage-1 map = 0%, reduce = 0%
2014-03-21 15:54:21,120 Stage-1 map = 0%, reduce = 0%
2014-03-21 15:55:55,130 Stage-1 map = 0%, reduce = 0%
2014-03-21 15:57:40,915 Stage-1 map = 0%, reduce = 0%
2014-03-21 15:59:28,636 Stage-1 map = 0%, reduce = 0%
Ended Job = job_201403211432_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://node1:60030/jobdetails.jsp?jobid=job_201403211432_0001
Examining task ID: task_201403211432_0001_m_000002 (and more) from job job_201403211432_0001
null
Exception in thread "Thread-46" java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:313)
at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:118)
at java.lang.Thread.run(Thread.java:722)
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

查看jobtracker页面中该job的信息发现有:

java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 5 more Caused by: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) ... 10 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 13 more Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121) ... 18 more Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple.(GenericUDTFJSONTuple.java:56) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:526) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:520) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.(FunctionRegistry.java:423) at org.apache.hadoop.hive.ql.exec.DefaultUDFMethodResolver.getEvalMethod(DefaultUDFMethodResolver.java:59) at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:154) at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:111) at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:141) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:227) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:543) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:100) ... 18 more Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.JsonFactory at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) ... 44 more

其中Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory是引起问题的原因。
二、原因:

Hadoop版本是0.20.2的,在$HADOOP_HOME/lib中不包含:

[hadoop@node2 lib]$ ll jackson*
-rw-rw-r-- 1 hadoop hadoop 227500 2012-01-21 09:24 jackson-core-asl-1.8.8.jar
-rw-rw-r-- 1 hadoop hadoop 17884 2012-01-21 09:26 jackson-jaxrs-1.8.8.jar
-rw-rw-r-- 1 hadoop hadoop 668564 2012-01-21 09:24 jackson-mapper-asl-1.8.8.jar
-rw-rw-r-- 1 hadoop hadoop 32353 2012-01-21 09:26 jackson-xc-1.8.8.jar

这四个包,因此会报错。

三、解决方案:

方法有两种:

1、在集群各节点上的hadoop的lib包增加相关的4个jar包,并重启集群。

2、hive prompt里输入:

hive> add jars /app/hive/lib/jackson-core-asl-1.8.8.jar jackson-jaxrs-1.8.8.jar jackson-mapper-asl-1.8.8.jar jackson-xc-1.8.8.jar
> ;
Added /app/hive/lib/jackson-core-asl-1.8.8.jar to class path
Added resource: /app/hive/lib/jackson-core-asl-1.8.8.jar
Added jackson-jaxrs-1.8.8.jar to class path
Added resource: jackson-jaxrs-1.8.8.jar
Added jackson-mapper-asl-1.8.8.jar to class path
Added resource: jackson-mapper-asl-1.8.8.jar
Added jackson-xc-1.8.8.jar to class path
Added resource: jackson-xc-1.8.8.jar

然后再执行select操作

Hive报错之java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory的更多相关文章

  1. 解决maven install报错:java.lang.NoClassDefFoundError: org/codehaus/plexus/compiler/util/scan/InclusionScanException

    问题:maven install时,报错:java.lang.NoClassDefFoundError: org/codehaus/plexus/compiler/util/scan/Inclusio ...

  2. 报错:java.lang.ClassNotFoundException: org.codehaus.jackson.map.JsonMappingException

    报错背景: 执行hdfs-mysql的job任务的时候报错. 报错现象: Exception has occurred during processing command Exception: org ...

  3. Weblogic 启动报错:java.lang.NoClassDefFoundError

    Weblogic 启动报错:java.lang.NoClassDefFoundError  ####<2015-6-17 下午03时30分47秒 CST> <Error> &l ...

  4. 【JUnit 报错】java.lang.NoClassDefFoundError: org/apache/logging/log4j/message/Message

    使用JUnit的时候,报错:java.lang.NoClassDefFoundError: org/apache/logging/log4j/message/Message 原因是因为项目中导入的架包 ...

  5. maven命令行创建web项目报错:java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils

    早上一上班就想新建一个web项目玩玩,没想到一敲命令创建就失败了,真是出师不利.各种折腾无果,当然我也可以用eclipse直接创建的,就是不甘心被这破问题给耍了.刚刚才发现问题原因,这个结果我也是醉了 ...

  6. jmeter之Java request报错:java.lang.NoClassDefFoundError: redis/clients/jedis/Jedis

    今天在学习Jmeter的java request,请求内容是连接redis并获取其中的一个字段值.结果在运行时报如下错误: 2018/05/24 13:08:20 ERROR - jmeter.thr ...

  7. tomcat启动报错:java.lang.NoClassDefFoundError

    tomcat启动加载spring配置文件时报错,找不到类GetBooksRequest,经排查实际上该类已经存在.后来发现日志里还有一句: This is very likely to create ...

  8. 就没有我遇不到的报错!java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/filter/Filter

    本来准备用HBase的Bulkload将HDFS的HFile文件导入到HBase的myuser2表中,用的是yarn jar的命令 yarn jar /export/servers/hbase-1.2 ...

  9. SpringBoot整合nacos启动报错:java.lang.NoClassDefFoundError: org/springframework/boot/context/properties/ConfigurationBeanFactoryMetadata

    报错信息 org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'nacosCo ...

随机推荐

  1. Android --设置Toast消失时间

    参考博客:Android开发,Toast重复显示(显示时间过长)解决方法 package com.dr.app.drseamoniter.toast; import android.content.C ...

  2. [ROS] slam_gmapping

    slam_gmapping节点 1)slam_gmapping 节点在sensor_msgs/LaserScan消息内获取数据并建立地图 map(nav_msgs/OccupancyGrid).该地图 ...

  3. UIPageControl

    UIPageControl 1.    numberOfPages // 设置有多少页默认为0 // 2) 设置页数 [pageControl setNumberOfPages:kImageCount ...

  4. Java- 基本封装

    package test.studet_manager; public class Student { static int num = 100; // 编号-->唯一的 private int ...

  5. Android的LinearLayout中的权重android:layout_weight

    当前EditText和Button部件只是适应了他们各自内容的大小,如下图所示: 这样设置对按钮来说很合适,但是对于文本框来说就不太好了,因为用户可能输入更长的文本内容.因此如果能够占满整个屏幕宽度会 ...

  6. .Net Framework 4.5.2 on Windows 10

    I was using Visual Studio 2013 to create a new solution, could not select ".NET Framework 4.5.2 ...

  7. transient关键字的含义

    transient java语言的关键字,变量修饰符,如果用transient声明一个实例变量,当对象存储时,它的值不需要维持. Java的serialization提供了一种持久化对象实例的机制.当 ...

  8. SQL查询一个表的总记录数的方法

    一.简单查询语句 1. 查看表结构 SQL>DESC emp; 2. 查询所有列 SQL>SELECT * FROM emp; 3. 查询指定列 SQL>SELECT empmo, ...

  9. Java基础(50):二分法查找的非递归实现和递归实现(完整代码可运行,参考VisualGO理解更佳)

    一.概念 二分查找算法也称折半查找,是一种在有序数组中查找某一特定元素的搜索算法. 二.算法思想 搜素过程从数组的中间元素开始,如果中间元素正好是要查找的元素,则搜素过程结束:如果某一特定元素大于或者 ...

  10. Java基础(37):Java中日期的显示与格式定值----Date与SimpleDateFormat的试用

    使用 Date 和 SimpleDateFormat 类表示时间 在程序开发中,经常需要处理日期和时间的相关数据,此时我们可以使用 java.util 包中的 Date 类.这个类最主要的作用就是获取 ...