问题:

将MongoDB数据导入Hive,按照https://blog.csdn.net/thriving_fcl/article/details/51471248文章,在hive建外部表与mongodb做映射后,执行后出现

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON

建表语句如下:

CREATE EXTERNAL TABLE mongotohive
  id string,
  userid string,
  age bigint,
  status string
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","userid":"user_id","age":"age","status":"status"}')
TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/mydb.users'); 

mongodb 数据如下:

db.users.find()
{ "_id" : ObjectId("5b456e33a93daf7ae53e6419"), "user_id" : "abc123", "age" : 58, "status" : "D" }
{ "_id" : ObjectId("5b45705ca93daf7ae53e8b2a"), "user_id" : "bcd001", "age" : 45, "status" : "C" }

解决方案:

将mongo-hadoop-core-2.0.0.jar、mongo-hadoop-hive-2.0.0.jar、mongo-java-driver-3.7.1.jar三个jar包放到hive的lib文件夹下后,再次运行成功。如下:

hive> CREATE EXTERNAL TABLE mongotohive
    > ( 
    >   id string,
    >   userid string,
    >   age bigint,
    >   status string
    > )
    > STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
    > WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","userid":"user_id","age":"age","status":"status"}')
    > TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/mydb.users');
OK
Time taken: 1.431 seconds
hive> select * from mongotohive;
OK
5b456e33a93daf7ae53e6419        abc123  58      D
5b45705ca93daf7ae53e8b2a        bcd001  45      C
Time taken: 0.601 seconds, Fetched: 2 row(s)
hive> 

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/util/JSON的更多相关文章

  1. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: An exception was thrown while adding/validating class(es) :

    在hive命令行创建表时报错: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. ...

  2. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me

    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(me ...

  3. Hive的Shell里hive> 执行操作时,出现FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask错误的解决办法(图文详解)

    不多说,直接上干货! 这个问题,得非 你的hive和hbase是不是同样都是CDH版本,还是一个是apache版本,一个是CDH版本. 问题详情 [kfk@bigdata-pro01 apache-h ...

  4. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

    hive启动后,出现以下异常 hive> show databases; FAILED: Error / failed on connection exception: java.net.Con ...

  5. hive_异常_01_(未解决)FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)V

    一.如果出现如下错误需要编译源码 需要重新编译Hbase-handler源码 步骤如下: 准备Jar包: 将Hbase 中lib下的jar包和Hive中lib下的jar包全部导入到一起. 记得删除里面 ...

  6. hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误

    hive遇到FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误 起因 ...

  7. Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/

    Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/apache/hadoop/hbase/ ...

  8. 解决hiveserver2报错:java.io.IOException: Job status not available - Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

    用户使用的sql: select count( distinct patient_id ) from argus.table_aa000612_641cd8ce_ceff_4ea0_9b27_0a3a ...

  9. java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

    执行Hive查询: Console是这样报错的 java.sql.SQLException: Error from org.apache.hadoop.hive.ql.exec.mr.MapRedTa ...

随机推荐

  1. kettle的系列教程

    kettle--window开发环境和linux运行环境的迁移 kettle的输入输出组件和脚本组件 kettle--变量的使用 kettle的资源库 kettle的job kettle的hello ...

  2. oracle-gi安装

    ############################################### 在cs6.5,cs7.2上安装grid11.2.0.1 和database11.2.0.1还是有问题 1 ...

  3. Mongo导出mongoexport和导入mongoimport介绍

    最近爬取mobike和ofo单车数据,需要存储在csv文件中,因为设计的程序没有写存储csv文件的方法,为了偷懒所以就继续存储到了MongoDB中.恰好MongoDB支持导出的数据可以是csv文件和j ...

  4. docker: Error response from daemon: driver failed programming external connectivity on endpoint jolly_shockley

    1. docker 启动image错误 docker: Error response from daemon: driver failed programming external connectiv ...

  5. RedHat7局域网下安装离线Ambari

    1 Ambari介绍.安装与应用案例介绍 1.1 Ambari Ambari 跟 Hadoop 等开源软件一样,也是 Apache Software Foundation 中的一个项目,并且是顶级项目 ...

  6. java的几种对象(po,dto,dao等)

    j2ee中,经常提到几种对象(object),理解他们的含义有助于我们更好的理解面向对象的设计思维.     POJO(plain old java object):普通的java对象,有别于特殊的j ...

  7. The Best KPIs to Use in Your Company

    Here is a list of key performance indicators (KPIs) that should be used in contact centres, alongsid ...

  8. 运动目标检测中基于HSV空间的阴影去除算法

    在运动目标检测中,常常会出现由于光线被遮挡,或场景其他物体的遮挡,在目标附近或场景里出现阴影,阴影的出现对后期目标的正确分割与处理带了很大的不便.如今,国内外已有不少文献来研究这个问题,并且提出了各种 ...

  9. InfluxDB 的UTC时间问题与简单的持续查询语句

    原文:https://blog.csdn.net/Vblegend_2013/article/details/80904275 最近项目中使用了时序数据库InfluxDB 各方性能也是蛮强大的.但是唯 ...

  10. ALGO-22_蓝桥杯_算法训练_数的划分(DP)

    问题描述 将整数n分成k份,且每份不能为空,任意两份不能相同(不考虑顺序). 例如:n=,k=,下面三种分法被认为是相同的. ,,; ,,; ,,; 问有多少种不同的分法. 输入格式 n,k 输出格式 ...