Spark Streaming从flume 中使用Poll拉取数据时,报如下错误:

Error while processing transaction.
java.lang.IllegalStateException: begin() called when transaction is OPEN!
  at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
  at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:114)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor$$anonfun$populateEvents$1.apply(TransactionProcessor.scala:113)
  at scala.Option.foreach(Option.scala:236)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243)
  at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43)
  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:745)
2020-04-08 05:20:15,120 (Spark Sink Processor Thread - 10) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Spark was unable to successfully process the events. Transaction is being rolled back.
2020-04-08 05:20:15,120 (New I/O worker #1) [WARN - org.apache.spark.streaming.flume.sink.Logging$class.logWarning(Logging.scala:59)] Received an error batch - no events were received from channel!

报错原因:  

  flume的lib目录下的scala-library.xx jar 版本有问题。

解决办法:

1、将flume的lib目录下的 scala-library-2.10.5.jar 删除或者重命名。

2、建议上传 scala-library-2.11.x.jar 包。

如下图:

3、附上 scala-library-2.11.8.jar ,需要请自提。

  [百度网盘:]
  链接:https://pan.baidu.com/s/1GsNwQyt89pobWHWkZO52jw

  提取码:41ik

Error while processing transaction.java.lang.IllegalStateException: begin() called when transaction is OPEN!的更多相关文章

  1. flume-sink报错 java.lang.IllegalStateException: close() called when transaction is OPEN - you must either commit or rollback first

    1. 确认代码无误(根据情况修改,表示若获得不了数据不会自动commit或者rollback): Event event = channel.take(); if (event == null) { ...

  2. java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

    java.lang.IllegalStateException: Not allowed to create transaction on sharedEntityManager - use Spri ...

  3. [spring cloud] [error] java.lang.IllegalStateException: Only one connection receive subscriber allowed.

    前言 最近在开发api-gateway的时候遇到了一个问题,网上能够找到的解决方案也很少,之后由公司的大佬解决了这个问题.写下这篇文章记录一下解决方案.希望可以帮助到更多的人. 环境 java版本:8 ...

  4. java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA transaction with specified [javax.transaction.TransactionManager] required

    错误信息: java.lang.IllegalStateException: Active Spring transaction synchronization or active JTA trans ...

  5. Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: aidl is missing from '/Users/renguodong/Library/Android/sdk/build-tools/26.0.2/aidl'

    错误信息:Error:Execution failed for task ':app:compileDebugAidl'. > java.lang.IllegalStateException: ...

  6. cause: java.lang.IllegalStateException: Serialized class com.taotao.pojo.TbItem must implement java.io.Serializable

    HTTP Status 500 - Request processing failed; nested exception is com.alibaba.dubbo.rpc.RpcException: ...

  7. nested exception is java.lang.IllegalStateException: Cannot forward after response has been committed

    org.springframework.web.util.NestedServletException: Request processing failed; nested exception is ...

  8. myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).

    把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...

  9. java.lang.IllegalStateException: getOutputStream() has already been called for this response

    ERROR [Engine] StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw exceptionjava.lang ...

随机推荐

  1. #AcWing系列课程Level-2笔记——5.高精度“+”算法

    高精度"+"算法 编写高精度"+",记住下面的过程,代码也就游刃有余了! 1.首先我们要明白大整数是如何存储的? 2.其次存储完,如何运算? 高精度" ...

  2. 使用SharpDevelop配合MonoGame进行游戏开发

    SharpDevelop是一款开源的轻量级IDE,它支持众多的语言及项目开发.可以看看支持的项目. 程序本体仅十几MB,打开项目速度飞快. 目前SharpDevelop最高支持C# 5.0,.NET ...

  3. Linux下MongoDB单实例的安装和配置详解

    推荐网站 MongoDB官网:http://www.mongodb.org/ MongoDB学习网站:http://www.runoob.com/mongodb 一.创建MongoDB的资源目录和安装 ...

  4. C语言程序设计(十二) 结构体和共用体

    第十二章 结构体和共用体 当需要表示复杂对象时,仅使用几个基本数据类型显然是不够的 根本的解决方法是允许用户自定义数据类型 构造数据类型(复合数据类型)允许用户根据实际需要利用已有的基本数据类型来构造 ...

  5. jupyternotebook安装

    本篇阅读目录 一.Jupyter notebook环境安装 二.爬虫介绍 回到顶部 一.Jupyter notebook环境安装 1.Anaconda 以及 安装步骤 Anaconda指的是一个开源的 ...

  6. PBM error occurred during PreCloneCheckCallback: 由于目标计算机积极拒绝,无法连接

    问题如下: 迁移存储和主机的时候发生错误,错误如下: 出现了常规系统错误: PBM error occurred during PreCloneCheckCallback: 由于目标计算机积极拒绝,无 ...

  7. 记一次nor flash固件烧录速度优化

    背景 某个方案使用的是spinor作为存储介质,每次烧录新固件都耗时数分钟,为了提高效率,需要对其进行优化. 分析流程 基本流程 当前烧录流程,有一个可选步骤,全盘擦除,这个步骤耗时达数分钟.不过这是 ...

  8. 【译】HTTP/2 为更快速的网站而生

    最近在做一个内部的服务对接,使用了Google的gRPC框架,gRPC是基于HTTP/2和protocol buffers实现的,所以额外去了解了一下HTTP/2,找到这这边文章.这篇文章虽然是写于2 ...

  9. Journal of Proteome Research | 人类牙槽骨蛋白的蛋白质组学和n端分析:改进的蛋白质提取方法和LysargiNase消化策略增加了蛋白质组的覆盖率和缺失蛋白的识别 | (解读人:卜繁宇)

    文献名:Proteomic and N-Terminomic TAILS Analyses of Human Alveolar Bone Proteins: Improved Protein Extr ...

  10. Error while importing sbt project:--创建sbt项目导入文件出错

    错误截图如下: Error while importing sbt project: List([info] Loading global plugins from C:\Users\RYJ\.sbt ...