Spark Streaming从flume 中使用Poll拉取数据时,报如下错误:

Error while processing transaction.
java.lang.IllegalStateException: begin() called when transaction is OPEN!
  at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
  at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:114)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:113)
  at scala.Option.foreach(Option.scala:236)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43)
  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:745)
2020-04-08 05:20:15,120 (Spark Sink Processor Thread - 10) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Spark was unable to successfully process the events. Transaction is being rolled back.
2020-04-08 05:20:15,120 (New I/O worker #1) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Received an error batch - no events were received from channel!

报错原因:  

  flume的lib目录下的scala-library.xx jar 版本有问题。

解决办法:

1、将flume的lib目录下的 scala-library-2.10.5.jar 删除或者重命名。

2、建议上传 scala-library-2.11.x.jar 包。

如下图:

3、附上 scala-library-2.11.8.jar ,需要请自提。

  [百度网盘:]
  链接:https://pan.baidu.com/s/1GsNwQyt89pobWHWkZO52jw

  提取码:41ik

Error while processing transaction.java.lang.IllegalStateException: begin() called when transaction is OPEN!的更多相关文章

  1. flume-sink报错 java.lang.IllegalStateException: close() called when transaction is OPEN - you must either commit or rollback first

    1. 确认代码无误(根据情况修改,表示若获得不了数据不会自动commit或者rollback): Event event = channel.take(); if (event == null) { ...

  2. java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

    java.lang.IllegalStateException: Not allowed to create transaction on sharedEntityManager - use Spri ...

  3. [spring cloud] [error] java.lang.IllegalStateException: Only one connection receive subscriber allowed.

    前言 最近在开发api-gateway的时候遇到了一个问题,网上能够找到的解决方案也很少,之后由公司的大佬解决了这个问题.写下这篇文章记录一下解决方案.希望可以帮助到更多的人. 环境 java版本:8 ...

  4. java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA transaction with specified [javax.transaction.TransactionManager] required

    错误信息: java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA trans ...

  5. Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: aidl is missing from '/Users/renguodong/Library/Android/sdk/build-tools/26.0.2/aidl'

    错误信息:Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: ...

  6. cause: java.lang.IllegalStateException: Serialized class com.taotao.pojo.TbItem must implement java.io.Serializable

    HTTP Status 500 - Request processing failed; nested exception is com.alibaba.dubbo.rpc.RpcException: ...

  7. nested exception is java.lang.IllegalStateException: Cannot forward after response has been committed

    org.springframework.web.util.NestedServletException: Request processing failed; nested exception is ...

  8. myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).

    把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...

  9. java.lang.IllegalStateException: getOutputStream() has already been called for this response

    ERROR [Engine] StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw exceptionjava.lang ...

随机推荐

  1. deepin中安装pycharm过程

    安装过程真的超级简单!一遍就会! 1.下载pycharm(下载地址):https://www.jetbrains.com/pycharm/ 2.在网址中找到对应的版本:在deepin中选择linux版 ...

  2. PAT资料,持续更新中~~~愿诸君共勉

    <算法笔记>胡凡著,<算法笔记-上机实战训练指南>胡凡著 <经典算法大全> <C陷阱与缺陷> <C程序设计语言-K&R> 链接:ht ...

  3. 关于使用Binlog和canal来对MySQL的数据写入进行监控

    先说下Binlog和canal是什么吧. 1.Binlog是mysql数据库的操作日志,当有发生增删改查操作时,就会在data目录下生成一个log文件,形如mysql-bin.000001,mysql ...

  4. 一起了解 .Net Foundation 项目 No.15

    .Net 基金会中包含有很多优秀的项目,今天就和笔者一起了解一下其中的一些优秀作品吧. 中文介绍 中文介绍内容翻译自英文介绍,主要采用意译.如与原文存在出入,请以原文为准. NUnit Test Fr ...

  5. 必备技能二、es6

    一.ES6模块 ES6 引入了模块化,其设计思想是在编译时就能确定模块的依赖关系,以及输入和输出的变量. ES6 的模块化分为导出(export) @与导入(import)两个模块. 特点 ES6 的 ...

  6. 项目测试中发现产品bug怎么办

    我所在的产品线,并非公司最大最强的产品 甚至为了推广我们这个产品,一般会拿给客户先免费试用 而在试用之前,是要经过一番通测的,测得很急,测得很快 所以产品bug非常多 那么在测试项目的时候,自然会发现 ...

  7. 使用VS2017进行Python代码的编写并打印出九九乘法表

    我们来盘一盘怎么使用VS2017进行python代码的编写并打印出九九乘法表. 使用Visual Studio 2017进行Python编程不需要太复杂的工作,只需要vs2017安装好对Python的 ...

  8. Simulink仿真入门到精通(十五) Simulink在流程工业中的仿真应用

    15.1 工业乙醇生产与计算机仿真 乙醇作为可再生清洁能源不仅可以代替四乙基铅作为汽油的防爆剂,还可以制造汽油醇.这一巨大的潜在需求促使人们去寻找提高乙醇工业生产率的途径,使人们着手于发酵工程的研究. ...

  9. Block详解一(底层分析)

    本篇博客不再讲述Block的基本定义使用,最近而是看了很多的block博客讲述的太乱太杂,所以抽出时间整理下block的相关底层知识,在讲述之前,提出几个问题,如果都可以回答出来以及知道原理,大神绕过 ...

  10. HTML5 history-hash 随机选择彩票

    <!doctype html> <html> <head> <meta charset="utf-8"> <title> ...