sqoop抽数报错 dolphinscheduler

2024-11-09

解决sqoop抽数报错：IO Error: Connection reset

遇到的问题:进行sqoop抽数时,虽然能成功执行,但是过程中有很多这样的信息 19/11/20 15:17:11 INFO mapreduce.Job: Task Id : attempt_1573713322094_0282_m_000002_0, Status : FAILEDError: java.lang.RuntimeException: java.lang.RuntimeException: java.sql.SQLRecoverableException: IO Error: Co

Sqoop 抽数报错： java.io.FileNotFoundException: File does not exist

Sqoop 抽数报错: java.io.FileNotFoundException: File does not exist 一.错误详情 2019-10-17 20:04:49,080 INFO [IPC Server handler 20 on 45158] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1567429685851_474405_m_000001_0: Err

解决sqoop导入报错：Caused by: java.sql.SQLException: Protocol violation

报错信息: -- ::, INFO [main] org.apache.hadoop.mapred.MapTask: Ignoring exception during close for org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader@7555b920 java.io.IOException: java.sql.SQLException: Protocol violation at org.apache.sqoop.mapred

sqoop mysql--->hive 报错（ERROR tool.ImportTool: Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf）

ERROR tool.ImportTool: Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf sqoop从mysql导入到hive报错: 18/08/22 13:30:53 ERROR tool.ImportTool: Import failed: java.io.IOException: java.lang.ClassNotFou

创建Sqoop作业,报错Exception in thread "main" java.lang.NoClassDefFoundError: org/json/JSONObject

WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. Exception in thread "main" java.lang.NoClassDefFoundError: org/json/JSONObject at org.apache.sqoop.util.SqoopJsonUtil.getJsonStrin

sqoop执行job报错（org/json/JSONObject）

NoClassDefFoundError: org/json/JSONObject: [root@hadoop2 ~]# sqoop job --create myjob7 -- import --connect jdbc:mysql://192.168.122.15:3306/company --username hivee --password-file file:///root/a.txt --table erp_admin_article --target-dir '/work/tes

解决sqoop 导入oracle表时 --split-by参数为日期类型时的报错：ORA-01861: literal does not match format string

报错栈: -- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: select "JFRQ","ZYH","FYKS","KSSE","YBJE","YPJE","ZJJE" from BSHIS."DEPCS_T_DEPCS_BRMXCOST_DAY

解决sqoop报错：java.lang.OutOfMemoryError: Java heap space

报错栈: -- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: = ) AND ( = ) -- ::, INFO [Thread-] org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false -- ::, FATAL [main] org.apache

解决Sqoop报错Could not load db driver class: com.intersys.jdbc.CacheDriver

报错栈: // :: INFO tool.CodeGenTool: Beginning code generation // :: ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.intersys.jdbc.CacheDriver java.lang.RuntimeException: Could not load db

使用sqoop从Oracle或mysql抽取数据到HDFS遇到的报错及解决

一.参考文档: 1.https://www.rittmanmead.com/blog/2014/03/using-sqoop-for-loading-oracle-data-into-hadoop-on-the-bigdatalite-vm/ 2.http://www.cnblogs.com/bjtu-leefon/archive/2013/06/28/3160549.html 二.使用脚本 ----sqoop import zdsd 使用注意: 1.执行节点 Sqoop_home/lib 里添

排查sqoop报错：Error running child : java.lang.OutOfMemoryError: Java heap space

报错栈: -- ::, INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: = AND = -- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Working on split: = AND = -- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing

解决sqoop报错Invalid number; item = ITEM_UNICODE

报错栈: java.sql.SQLException: Invalid number; item = ITEM_UNICODE at com.intersys.jdbc.SysList.getInt(SysList.java:) at com.intersys.jdbc.CacheResultSet.getInt(CacheResultSet.java:) at org.apache.sqoop.lib.JdbcWritableBridge.readInteger(JdbcWritableBri

解决sqoop报错：SQLServerException: 将字符串转换为 uniqueidentifier 时失败。

报错栈: Error: java.io.IOException: Cannection handler cannot recover failure: at org.apache.sqoop.mapreduce.db.SQLServerDBRecordReader.nextKeyValue(SQLServerDBRecordReader.java:) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(

sqoop从mysql导数据到hive报错：Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

背景使用sqoop从mysql导数据到hive,从本地服务器是可以访问mysql的(本地服务器是hadoop集群的一个datanode),但是sqoop导数据的时候依然连接不上mysql 报错如下: 从报错可以看出,是数据库连接失败,很常见的问题,但是从本地是可以直连mysql的. 因为sqoop导数据的时候,默认会启动4个map task,这4个map task会随机启动在不动的datanode上,所以在想,是不是因为其他节点没有权限访问mysql导致. 但是需要先搞清楚,sqoop在抽取数

报错：Exception: org.apache.sqoop.common.SqoopException Message: DRIVER_0002:Given job is already running - Job with id 1

报错背景: 创建完成job之后,执行job的时候报错. 报错现象: Exception: org.apache.sqoop.common.SqoopException Message: CLIENT_0001:Server has returned exception Stack trace: at org.apache.sqoop.client.request.ResourceRequest (ResourceRequest.java:) at org.apache.sqoop.client.

报错：org.apache.sqoop.common.SqoopException Message: CLIENT_0001:Server has returned exception NoClassDefFoundError: org/codehaus/jackson/map/JsonMappingException

报错背景: CDH集成sqoop2服务之后,创建好link和job之后,执行job的时候报错. 报错现象: sqoop:> start job -j Exception has occurred during processing command Exception: org.apache.sqoop.common.SqoopException Message: CLIENT_0001:Server has returned exception - <html>< - Error