报错栈:

-- ::, INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: = AND =
-- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Working on split: = AND =
-- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: select "EXTEND3","EXTEND2","EXTEND1","MEMO","OPER_DATE","OPER_CODE","FILE_CONTENT","FILE_NAME","INPATIENT_NO","ID" from HIS_SDZL."MDT_FILE" tbl where ( = ) AND ( = )
-- ::, INFO [Thread-] org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
-- ::, FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:)
at java.lang.StringBuffer.append(StringBuffer.java:)
at java.util.regex.Matcher.appendReplacement(Matcher.java:)
at java.util.regex.Matcher.replaceAll(Matcher.java:)
at java.lang.String.replaceAll(String.java:)
at QueryResult.readFields(QueryResult.java:)
at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:)
at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:)
at org.apache.hadoop.mapred.YarnChild$.run(YarnChild.java:)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:)

调小fetchsize参数也不能解决,那问题很可能是某行数据占用空间很大。根据Sqoop生成的导入表对应的实例化类QueryResult.java的244行可定位到报错列是FILE_CONTENT,是个二进制列, 然后查询原库,果然最大的列长达到180M:

ps: 怎么用标准的sql语句查询 blob字段的大小?
blob字段有好多种。如果是9i的简单的blob字段则应该是 length,或者lengthb也可。实在不行可以用 dbms_lob.getlength()

排查sqoop报错:Error running child : java.lang.OutOfMemoryError: Java heap space的更多相关文章

  1. 解决sqoop报错:java.lang.OutOfMemoryError: Java heap space

    报错栈: -- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: = ) AND ( = ) ...

  2. 【Xamarin报错】 COMPILETODALVIK : UNEXPECTED TOP-LEVEL error java.lang.OutOfMemoryError: Java heap space

    Xamarin Android 编译报错: COMPILETODALVIK : UNEXPECTED TOP-LEVEL error java.lang.OutOfMemoryError: Java ...

  3. 执行Hive出现Error running child : java.lang.OutOfMemoryError: Java heap space错误

    具体错误日志如下: 2018-05-11 15:16:49,429 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running chi ...

  4. 【IntellJ IDEA】idea启动测试类报错Error running 'Test1.test': Command line is too long. Shorten command line for Test1.test or also for JUnit default configuration.

    idea启动测试类报错 Error running 'Test1.test': Command line is too long. Shorten command line for Test1.tes ...

  5. 运行springboot项目报错 Error running 'ResourceApplication': Command line is too long. Shorten comma

    方法1 IDEA 运行报错:Error running '***': Command line is too long 技术标签: IDEA     Error running 'Test': Com ...

  6. Tomcat 启动项目报错 java.lang.OutOfMemoryError: Java heap space

    近日使用myeclipse 部署web项目,启动tomcat时报错: SEVERE: Error waiting for multi-thread deployment of directories ...

  7. Linux运行Java出现“Exception in thread "main" java.lang.OutOfMemoryError: Java heap space”报错

    在运行如下程序时出现“Exception in thread "main" java.lang.OutOfMemoryError: Java heap space”报错: java ...

  8. mycat 报错 java.lang.OutOfMemoryError: Java heap space

    今天排查mysql的错误日志发现  wrapper.log  中有如下错误日志 INFO   | jvm 1    | 2019/10/20 12:52:31 | java.lang.OutOfMem ...

  9. eclipse内存溢出报错:java.lang.OutOfMemoryError:Java heap space

    今天执行了一个比較大的程序,处理的数据达到126MB数据,将数据导入数据库中,用eclipse 来訪问时候,总是出现java.lang.OutOfMemoryError:Java heap space ...

随机推荐

  1. java.lang.reflect.UndeclaredThrowableExceptionjiang

    实例包含由调用处理程序抛出的经过检查的未声明异常,可以使用 getUndeclaredThrowable() 方法获取 String msg = null; if (e instanceof Unde ...

  2. electron 使用中的注意事项

    一.ELECTRON引用JQUERY.JS electron不能像正常的html文件引用jq.js那样(为嘛不造),elecron引用jq.js的方式为: <script>window.$ ...

  3. jquery validate表单验证插件的基本使用方法及功能拓展

    1 表单验证的准备工作 在开启长篇大论之前,首先将表单验证的效果展示给大家.    1.点击表单项,显示帮助提示 2.鼠标离开表单项时,开始校验元素 3.鼠标离开后的正确.错误提示及鼠标移入时的帮助提 ...

  4. 分页查询和redis

    问题 我在做论坛的是时候遇到了如下的问题.论坛里可以有很多的主题topic,每个topic对应到很多回复reply.现在要查询某个topic下按照replyTime升序排列的第pageNo页的repl ...

  5. Xml配置文件属性的说明

    Xml配置文件属性的说明: <bean id="TheAction" ⑴ class="net.xiaxin.spring.qs.UpperAction" ...

  6. 【BZOJ 3144】 3144: [Hnoi2013]切糕 (最小割模型)

    3144: [Hnoi2013]切糕 Time Limit: 10 Sec  Memory Limit: 128 MBSubmit: 1764  Solved: 965 Description Inp ...

  7. Dalvik 虚拟机 jvm 区别

    韩梦飞沙  韩亚飞  313134555@qq.com  yue31313  han_meng_fei_sha dalvik 基于 寄存器, jvm基于 栈. 寄存器,编译时间会更短. dalvik ...

  8. HDU 5514 Frogs(容斥原理)

    [题目链接] http://acm.hdu.edu.cn/showproblem.php?pid=5514 [题目大意] m个石子围成一圈,标号为0~m-1,现在有n只青蛙,每只每次跳a[i]个石子, ...

  9. 【二分答案】BZOJ2016-Chocolate Eating

    [题目大意] n块巧克力,每次吃可以增加ai点快乐,每天早晨睡觉起来快乐值会减半,求如何使d天睡觉前的最小快乐值最大. [思路] 二分每天的最小快乐值,只要没有达到快乐值就继续吃. 不知道为什么了WA ...

  10. bzoj 1030

    dp[i][j] 表示,在AC自动机中,从根节点开始,走了i条边,并且经过的点不包含危险节点,走到了j节点的路径数. 收获: 1.正难则反 2.一个字符串不包含给定pattern中的任何一个,则该字符 ...