HamaWhite 原创，转载请注明出处。欢迎大家增加Giraph
技术交流群： 228591158

1. 近日用Giraph跑大数据的SSSP时，遇到例如以下错误：

org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449)
at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506) at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60)
at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117) at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) FATAL graph.GraphMapper (GraphMapper.java:uncaughtException(113)) - uncaughtException: OverrideExceptionHandler
on thread org.apache.giraph.master.MasterThread, msg = org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting... java.lang.IllegalStateException: org.apache.hadoop.mapred.Counters$CountersExceededException:
Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.giraph.master.MasterThread.run(MasterThread.java:181) Caused by: org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters -
Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60) at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117)
at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) INFO zk.ZooKeeperManager (ZooKeeperManager.java:run(655)) - run: Shutdown hook started. WARN zk.ZooKeeperManager (ZooKeeperManager.java:run(658)) - onlineZooKeeperServers: Forced a shutdown
hook kill of the ZooKeeper process. INFO zookeeper.ClientCnxn (ClientCnxn.java:run(1159)) - Unable to read additional data from server sessionid 0x145750f19970000, likely server has closed socket, closing socket connection and attempting reconnect INFO zk.ZooKeeperManager
(ZooKeeperManager.java:run(668)) - onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143 typically means killed).

2. 错误原因分析：该Job的Counters数量限制超过默认的120 。mapred-default.xml配置文件的默认例如以下：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>120</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

3. 改动配置文件mapred-site.xml 。加入例如以下内容：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>1200</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

4. 停止Hadoop服务 。

5. 把改动后的配置文件发送到集群中的每一个节点上。

6. 启动Hadoop服务。

完成！

Giraph执行报错，Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting...的更多相关文章

  1. 【mysql】 load local data infield 报错 ERROR 1148 (42000): The used command is not allowed with this MySQL version

    mysql> load data local infile '/Users/flint/learn/mysql/pet' into table bx_pet; 执行报错 ERROR 1148 ( ...

  2. php源码安装执行configure报错error: off_t undefined; check your library configuration

    php安装执行configure报错error: off_t undefined; check your library configuration vim /etc/ld.so.conf 添加如下几 ...

  3. zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理

    zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理 问题描述: 使用zeppli ...

  4. open数据库报错ERROR at line 1: ORA-03113: end-of-file on communication channel Process ID: 3880 Session ID: 125 Serial number: 3

    1.今天打开数据时,失败,报错 ERROR at line 1:ORA-03113: end-of-file on communication channelProcess ID: 3880Sessi ...

  5. linux使用wkhtmltopdf报错error while loading shared libraries:

    官网提示 linux需要这些动态库.depends on: zlib, fontconfig, freetype, X11 libs (libX11, libXext, libXrender) 在li ...

  6. 发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store

    发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store 昨晚上传项目到AppStore,报了这个错,纳尼! ...

  7. mysql报错"ERROR 1206 (HY000): The total number of locks exceeds the lock table size"的解决方法

    1. 问题背景         InnoDB是新版MySQL(v5.5及以后)默认的存储引擎,之前版本的默认引擎为MyISAM,因此,低于5.5版本的mysql配置文件.my.cnf中,关于InnoD ...

  8. 运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for reposit 今天给Centos通过rpm - ...

  9. 转：运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for repository: epel. Please ...

随机推荐

  1. 从Python的角度来看编码与解码

    导语: Python2和Python3中,因为默认字符集的不同而造成的麻烦,简直是程序员的梦魇!要彻底告别这个麻烦,就需要从本质上来理解编码和解码. 为什么要有编码? 对于不会英文的中国人来说,将英文 ...

  2. strncmp----c++库函数

    某天,蒜头君和花椰妹在公园里散步,走着走着,我的天!他们各自都捡到了一串漂亮的字符串,然而蒜头君好奇心比较重,他想知道自己的字符串在花椰妹的字符串中出现了多少次,例如花椰妹的字符串为abababa,蒜 ...

  3. CodeForces 586B Laurenty and Shop

    F - Laurenty and Shop Time Limit:1000MS     Memory Limit:262144KB     64bit IO Format:%I64d & %I ...

  4. mysql 5.6.20 数据库中文乱码解决方法

    1.打开mysql5.6安装目录下面的文件my-default.ini在[client]下面添加default-character-set=utf8(如果没有[client]自己手动添加).然后在[m ...

  5. ECMall 中URL体系的改造思路

    EC系列的产品都已停止更新很久了,但其对中国中小电商企业的影响无疑是巨大的.很多公司,都是直接拿来即改,改了即用. 但他们都有个问题,代码是比较传统的开发模式过来的,尤其ecshop.ECMall系统 ...

  6. root权限NPM全局安装(-g)仍会权限不够，认识下参数 --unsafe-perm

    环境说明 ganiks@ganiks-ubuntu-trusty-64:/ganiks/parse-server$ npm -v 6.5.0 ganiks@ganiks-ubuntu-trusty-6 ...

  7. Hive学习笔记——Hive中的分桶

    对于每一个表(table)或者分区, Hive可以进一步组织成桶,也就是说桶是更为细粒度的数据范围划分.Hive也是针对某一列进行桶的组织.Hive采用对列值哈希,然后除以桶的个数求余的方式决定该条记 ...

  8. 第二百一十七节，jQuery EasyUI，NumberSpinner(数字微调)组件

    jQuery EasyUI,NumberSpinner(数字微调)组件 学习要点: 1.加载方式 2.属性列表 3.事件列表 4.方法列表 本节课重点了解 EasyUI 中 NumberSpinner ...

  9. Container类是Component的子类，它也是一个抽象类，它允许其他的组件(Component)加入其中

    在AWT中,所有能在屏幕上显示的组件(component )对应的类,均是 抽象类 Component 的子类或子孙类. 这些类均可继承Component类的变量和方法. Container类是Com ...

  10. Storm集成Kafka编程模型

    原创文章,转载请注明: 转载自http://www.cnblogs.com/tovin/p/3974417.html 本文主要介绍如何在Storm编程实现与Kafka的集成 一.实现模型 数据流程: ...