Error:"MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. getCurrentNotificationEventId" occurs as HiveServer2 fails to start as it cannot connect to Metastore in HDP 3.0
SupportKB
Problem Description:
After upgrading to HDP 3.0, the HiveServer2 fails to start and the following error is displayed in the HiveServer2 logs:
- WARN [main]: metastore.RetryingMetaStoreClient (:()) -
- MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. getCurrentNotificationEventId
- org.apache.thrift.TApplicationException: Internal error processing get_current_notificationEventId
- at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
- ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
- ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_current_notificationEventId
- (ThriftHiveMetastore.java:5575) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_current_notificationEventId
- (ThriftHiveMetastore.java:5563) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getCurrentNotificationEventId
- (HiveMetaStoreClient.java:2723) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
- at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
- at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
- at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
- at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
- ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at com.sun.proxy.$Proxy40.getCurrentNotificationEventId(Unknown Source) ~[?:?]
- at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
- at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
- at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
- at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
- at org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke
- (HiveMetaStoreClient.java:2773) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at com.sun.proxy.$Proxy40.getCurrentNotificationEventId(Unknown Source) ~[?:?]
- at org.apache.hadoop.hive.metastore.messaging.EventUtils$MSClientNotificationFetcher.
- getCurrentNotificationEventId(EventUtils.java:73)
- ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.ql.metadata.events.NotificationEventPoll.<init>
- (NotificationEventPoll.java:103)
- ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.ql.metadata.events.NotificationEventPoll.initialize
- (NotificationEventPoll.java:59)
- ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:275)
- ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1063)
- ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hive.service.server.HiveServer2.access$1600(HiveServer2.java:140)
- ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1332)
- ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1176)
- ~[hive-service-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
- at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
- ~[?:1.8.0_112]
- at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
- at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
- at org.apache.hadoop.util.RunJar.run(RunJar.java:318) ~[hadoop-common-3.1.0.3.0.0.0-1634.jar:?]
- at org.apache.hadoop.util.RunJar.main(RunJar.java:232) ~[hadoop-common-3.1.0.3.0.0.0-1634.jar:?]
In hive-metastore.log, the following message is displayed at the same timeline:
- ERROR [pool-6-thread-200]: metastore.HiveMetaStore (HiveMetaStore.java:get_current_notificationEventId(7617))
- - Not authorized to make the get_current_notificationEventId call.
- You can try to disable metastore.metastore.event.db.notification.api.auth
- org.apache.hadoop.hive.metastore.api.MetaException: User hive is not allowed to perform this API call
- at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.authorizeProxyPrivilege
- (HiveMetaStore.java:7655)
- ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_current_notificationEventId(HiveMetaStore.java:7615)
- [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) ~[?:?]
- at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
- ~[?:1.8.0_112]
- at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
- at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
- [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
- [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at com.sun.proxy.$Proxy34.get_current_notificationEventId(Unknown Source) [?:?]
- at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_current_notificationEventId.getResult
- (ThriftHiveMetastore.java:18364)
- [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_current_notificationEventId.getResult
- (ThriftHiveMetastore.java:18349) [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
- [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
- [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
- [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at java.security.AccessController.doPrivileged(Native Method) [?:1.8.0_112]
- at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_112]
- at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1688)
- [hadoop-common-3.1.0.3.0.0.0-1634.jar:?]
- at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
- [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
- [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634]
- at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
- [?:1.8.0_112]
- at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
- [?:1.8.0_112]
- at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Cause:
The connection to Metastore from HiveServer2 fails after upgrading the cluster when hive.metastore.event.db.notification.api.auth property is set to true.
Solution:
This is a known issue (HIVE-19740).
Workaround In HDFS > Configs > Custom core-site.xml, add the following:
- hadoop.proxyuser.hive.hosts=HS2_HOST
- hadoop.proxyuser.hive.groups=*
Or
In Hive > Configs > Custom hive-site.xml, add the following:
- hive.metastore.event.db.notification.api.auth=false
Restart the required services.
For more details about this property, see HiveReplicationv2Development
About:
This article created by Hortonworks Support (Article: 000007703) on 2018-08-06 07:32
OS: All
Type: Cluster_Administration
Version: H3.0.0
Error:"MetaStoreClient lost connection. Attempting to reconnect (1 of 24) after 5s. getCurrentNotificationEventId" occurs as HiveServer2 fails to start as it cannot connect to Metastore in HDP 3.0的更多相关文章
- WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2017-05-19 13:32:14,933] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient) ...
- WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) java.net.ConnectException: Connection refused
1.启动kafka的脚本程序报如下所示的错误: [hadoop@slaver1 script_hadoop]$ kafka-start.sh start kafkaServer... [-- ::,] ...
- hadoop mapreduce 写入hbase报错 Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
现象:map任务构造数据正常,reduce任务,开始也正常,速度很快 ,在hbase 的管理界面,可以看到,5W以上的请求数 当reduce 执行到 70% 左右的时候,就堵住了,查看yarn的web ...
- Linux下启动tomcat报错,WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException:
tomcat启动完了之后,一直不停的打印这种错误信息,看表面上,应该是zk节点下的数据是空的,连接不上服务,所以一直在尝试连接,然后一直又连不上: 完整的错误信息: 407662 [usf-ZooKe ...
- R语言保存文件 Error in save error writing to connection
Error in save(filtered, file = paste(sampleName, "filtered", sep = "_")) : err ...
- urllib2.URLError: <urlopen error [Errno 104] Connection reset by peer>
http://www.dianping.com/shop/8010173 File "综合商场1.py", line 152, in <module> httpC ...
- STM8学习 无法仿真原因Starting debug session... -> Emulator reset (usb://usb)... ** Connection error (usb://usb): swim error [30200]: st-link connection error
刚调试程序时,STlink总是连不上,一直提示: Starting debug session...-> Emulator reset (usb://usb)...** Connection e ...
- mysqldump导出报错"mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `file_storage` at row: 29"
今天mysql备份的crontab自动运行的时候,出现了报警,报警内容如下 mysqldump: Error 2013: Lost connection to MySQL server during ...
- Putty出现 Network error:Software caused connection abort
使用centos7.5 用Putty连接使用没多久就会出现 Network error:Software caused connection abort #修改sshd配置文件.修改3项配置即可 vi ...
随机推荐
- 关于Linux虚拟化技术KVM的科普 科普四(From humjb_1983)
另一组关于KVM的分析文档,虚拟化相关概念.KVM基本原理和架构一-概念和术语.KVM基本原理和架构二-基本原理.KVM基本原理及架构三-CPU虚拟化.KVM基本原理及架构四-内存虚拟化.KVM基本原 ...
- 使用nginx搭建https服务器
http://www.cnblogs.com/tintin1926/archive/2012/07/12/2587311.html 最近在研究nginx,整好遇到一个需求就是希望服务器与客户端之间传输 ...
- sublime text3汉化
注意在安装 sublime text3 时勾选Add to explorer context menu,这样在右键单击文件时就可以直接使用Sublime Text打开. 下载Package Contr ...
- Python hashlib 无法打印
# !/user/bin/python # -*- coding: utf-8 -*- import hashlib # 可提供MD5算法 , 防止内页被篡改 (若内页未被篡改, MD5的值是不变的) ...
- c# 如何找到项目中图片的相对路径
c# 如何找到项目中图片的相对路径 string path = System.Environment.CurrentDirectory;//非Web程序if (System.Environment.C ...
- 【强连通分量+概率】Bzoj2438 杀人游戏
Description 一位冷血的杀手潜入 Na-wiat,并假装成平民.警察希望能在 N 个人里面,查出谁是杀手. 警察能够对每一个人进行查证,假如查证的对象是平民,他会告诉警察,他认识的人, 谁是 ...
- [ZLXOI2015]殉国 数论 扩展欧几里得
题目大意:已知a,b,c,求满足ax+by=c (x>=0,y>=0)的(x+y)最大值与最小值与解的个数. 直接exgcd,求出x,y分别为最小正整数的解,然后一算就出来啦 #inclu ...
- LuoguP4234_最小差值生成树_LCT
LuoguP4234_最小差值生成树_LCT 题意: 给出一个无向图,求最大的边权减最小的边权最小的一棵生成树. 分析: 可以把边权从大到小排序,然后类似魔法森林那样插入. 如果两点不连通,直接连上, ...
- BZOJ3252: 攻略 可并堆
网上有很多人说用dfs序+线段树做...其实stl的堆可以...可并堆可以...很多奇奇怪怪的东西都能做... 可并堆比较好想...也比较好写... 分析: 首先,这是一个网络流做不了的题...数据太 ...
- 【爆料】-《伯明翰大学学院毕业证书》UCB一模一样原件
☞伯明翰大学学院毕业证书[微/Q:865121257◆WeChat:CC6669834]UC毕业证书/联系人Alice[查看点击百度快照查看][留信网学历认证&博士&硕士&海归 ...