问题描述

在HBase上运行MapReduce作业时，报如下异常：IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString

使用HBase环境如下：CDH5.0.1, HBase版本：0.96.1

问题原因

This isssue occurs because of an optimization introduced in HBASE-9867 that inadvertently introduced a classloader dependency. This affects both jobs using the -libjars option and "fat jar," jobs which package their runtime dependencies in a nested lib folder.

这个问题的发生是由于优化了HBASE-9867 引起的，无意间引进了一个依赖类加载器。它影响使用-libjars参数和使用 fat jar两种模式的job.

fat jar模式Hadoop的一个特殊功能：可以读取操作目录中/lib目录下包含的所有库的JAR文件，把运行job依赖的jar放在jar中的lib目录下。

解决方案

1、永久解决方案

cd /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop

ln -s /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar hbase-protocol-0.96.1.1-cdh5.0.0.jar

2: 临时解决方案

HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf hadoop  jar myjar.jar MyJobMainClass

详细报错信息

14/07/16 14:40:50 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch hbase:meta table: 
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=10, exceptions:
Wed Jul 16 14:32:49 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405,java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString
Wed Jul 16 14:32:52 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:32:57 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:33:07 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:33:28 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:34:08 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:35:49 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:37:29 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:39:10 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:40:50 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405,java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString

at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
        at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:705)
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:144)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1102)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1162)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1054)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1011)
        at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:326)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:192)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150)
        at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:206)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
        at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:455)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
        at mapreduce.ImportFromFile.run(ImportFromFile.java:228)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at mapreduce.ImportFromFile.main(ImportFromFile.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152)
        at mapreduce.Driver.main(Driver.java:29)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:897)
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest(RequestConverter.java:131)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1402)
        at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:701)
        at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:699)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
        ... 38 more
Exception in thread "main" java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:897)
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest(RequestConverter.java:131)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1402)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1176)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1054)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1011)
        at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:326)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:192)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150)
        at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:206)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
        at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:455)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
        at mapreduce.ImportFromFile.run(ImportFromFile.java:228)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at mapreduce.ImportFromFile.main(ImportFromFile.java:185)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152)
        at mapreduce.Driver.main(Driver.java:29)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

参考

https://issues.apache.org/jira/browse/HBASE-11118

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_releasenotes_hdp_2.0/content/ch_relnotes-hdpch_relnotes-hdp-2.0.9.0-knownissues-hbase.html

http://my.oschina.net/cloudcoder/blog/291486

ZeroCopyLiteralByteString cannot access superclass的更多相关文章

  1. shiro创建配置对象

    在执行 Factory<org.apache.shiro.mgt.SecurityManager> factory = new IniSecurityManagerFactory(&quo ...

  2. java.lang.IllegalAccessError: class javax.activation.SecuritySupport12 cannot access its superclass

    最近加入新的项目组,eclipse + tomcat7 + spring +ibatis + restful 遇到了这样的问题, 说是不能访问父类,我一开始以为是版本的原因,但是久经更改,错误依然,实 ...

  3. Scala access modifiers and qualifiers in detail

    来自:http://www.jesperdj.com/2016/01/08/scala-access-modifiers-and-qualifiers-in-detail/ Just like Jav ...

  4. 解决Unable to resolve superclass的问题

    之前在GITHUB上看到大蛋的高级设置,昨晚于是就拿来编译.. 结果真是个悲伤的故事,放手机上居然运行不了,这种时候只能看LOG了! 看LOG得出的信息如下: - ::): threadid=: in ...

  5. 【基于WinForm+Access局域网共享数据库的项目总结】之篇一：WinForm开发总体概述与技术实现

    篇一:WinForm开发总体概述与技术实现 篇二:WinForm开发扇形图统计和Excel数据导出 篇三:Access远程连接数据库和窗体打包部署 [小记]:最近基于WinForm+Access数据库 ...

  6. windows 部署 git 服务器报 Please make sure you have the correct access rights and the repository exists.错误

    这两天在阿里云上弄windows 服务器,顺便部署了一个git服务.根据网上教程一步步操作下来,最后在 remote远程仓库的时候提示 fatal: 'yourpath/test.git' does ...

  7. 【.net 深呼吸】连接Access数据库应注意的几点

    本地数据库可以有Y种选择,比如Sqlite.SQL Server Express.SQL Local DB.SQL Server CE.Access等,本文老周选用比较著名的Access本地数据库,在 ...

  8. ASP.NET OAuth：解决refresh token无法刷新access token的问题

    最近同事用iOS App调用Open API时遇到一个问题:在access token过期后,用refresh token刷新access token时,服务器响应"invalid_gran ...

  9. ASP.NET OAuth：access token的加密解密，client secret与refresh token的生成

    在 ASP.NET OWIN OAuth(Microsoft.Owin.Security.OAuth)中,access token 的默认加密方法是: 1) System.Security.Crypt ...

随机推荐

  1. 【Cocos2d实例教程一】xcode5下Cocos2d环境的搭建

    (转载请注明出处:http://blog.csdn.net/buptgshengod) 第一步,现在要安装集成环境xcode5,安装xcode5需要系统至少是os x 10.8.5. 第二步,下载co ...

  2. 理解Java的封装与接口

    1.封装,即保留有限的外部接口(interface),隐藏具体实施细节. 2.封装在生活中很常见.比如下面是一个充电电筒: 一个用户即使不看说明书,也可以猜到这个电筒的操作: 开关和充电.这个电筒用一 ...

  3. 【Android】 PopupWindow使用小结

        PopupWindow的很多用法网上比较多,我就不做过多解释了,只说下可能会遇到的问题,以及解决办法: 1.PopupWindow中的listview无响应 这个主要是因为show写在了set ...

  4. Qt Linguist介绍

    简介 Qt提供了一款优秀的支持Qt C++和Qt Quick应用程序的翻译工具.发布者.翻译者和开发者可以使用这款工具来完成他们的任务. 发布者:承担了全面发布应用程序的责任.通常,他们协调开发者和翻 ...

  5. POJ 3468 (线段树 区间增减) A Simple Problem with Integers

    这题WA了好久,一直以为是lld和I64d的问题,后来发现是自己的pushdown函数写错了,说到底还是因为自己对线段树理解得不好. 因为是懒惰标记,所以只有在区间分开的时候才会将标记往下传递.更新和 ...

  6. 【第七篇】bootstrap的3级菜单样式，支持母版页保留打开状态

    razor视图的,母版页 依旧不多说,直接上代码 <ul class="sidebar-menu"> @for (int i = 0; i < mList.Cou ...

  7. UVa 101 The Blocks Problem

    题意:给出从左到右放置的n块木块(从0开始编号),再给出四种操作,再给出相应的操作,输出操作结束后每一堆木块的情况. 学习的紫书,因为每一堆的木块数是在发生变化的,所以用vector. 然后就是模拟几 ...

  8. windowsUI的总结

    1,MFC 基于VC6.0的微软基础库 2,WPF 做绚丽界面一律用WPF,做一般绚丽界面用WinForm,做windows标准界面用MFC WPF也有个致命缺点,就是要.net framework支 ...

  9. LeetCode Binary Tree Maximum Path Sum 二叉树最大路径和（DFS）

    题意:给一棵二叉树,要求找出任意两个节点(也可以只是一个点)的最大路径和,至少1个节点,返回路径和.(点权有负的.) 思路:DFS解决,返回值是,经过从某后代节点上来到当前节点且路径和最大的值.要注意 ...

  10. 【转】Linux设备驱动之I/O端口与I/O内存

    原文网址:http://www.cnblogs.com/geneil/archive/2011/12/08/2281367.html 一.统一编址与独立编址 该部分来自于:http://blog.ch ...