HamaWhite 原创，转载请注明出处。欢迎大家增加Giraph
技术交流群： 228591158

1. 近日用Giraph跑大数据的SSSP时，遇到例如以下错误：

org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449)
at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506) at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60)
at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117) at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) FATAL graph.GraphMapper (GraphMapper.java:uncaughtException(113)) - uncaughtException: OverrideExceptionHandler
on thread org.apache.giraph.master.MasterThread, msg = org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting... java.lang.IllegalStateException: org.apache.hadoop.mapred.Counters$CountersExceededException:
Error: Exceeded limits on number of counters - Counters=120 Limit=120 at org.apache.giraph.master.MasterThread.run(MasterThread.java:181) Caused by: org.apache.hadoop.mapred.Counters$CountersExceededException: Error: Exceeded limits on number of counters -
Counters=120 Limit=120 at org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:315) at org.apache.hadoop.mapred.Counters.findCounter(Counters.java:449) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:559) at org.apache.hadoop.mapred.Task$TaskReporter.getCounter(Task.java:506)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.getCounter(TaskInputOutputContext.java:88) at org.apache.giraph.counters.HadoopCountersBase.getCounter(HadoopCountersBase.java:60) at org.apache.giraph.counters.GiraphTimers.getSuperstepMs(GiraphTimers.java:117)
at org.apache.giraph.master.MasterThread.run(MasterThread.java:127) INFO zk.ZooKeeperManager (ZooKeeperManager.java:run(655)) - run: Shutdown hook started. WARN zk.ZooKeeperManager (ZooKeeperManager.java:run(658)) - onlineZooKeeperServers: Forced a shutdown
hook kill of the ZooKeeper process. INFO zookeeper.ClientCnxn (ClientCnxn.java:run(1159)) - Unable to read additional data from server sessionid 0x145750f19970000, likely server has closed socket, closing socket connection and attempting reconnect INFO zk.ZooKeeperManager
(ZooKeeperManager.java:run(668)) - onlineZooKeeperServers: ZooKeeper process exited with 143 (note that 143 typically means killed).

2. 错误原因分析：该Job的Counters数量限制超过默认的120 。mapred-default.xml配置文件的默认例如以下：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>120</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

3. 改动配置文件mapred-site.xml 。加入例如以下内容：

<property>

          <name>mapreduce.job.counters.limit</name>

          <value>1200</value>

          <description>Limit on the number of counters allowed per job. </description>

     </property>

4. 停止Hadoop服务 。

5. 把改动后的配置文件发送到集群中的每一个节点上。

6. 启动Hadoop服务。

完成！

Giraph执行报错，Error: Exceeded limits on number of counters - Counters=120 Limit=120, exiting...的更多相关文章

  1. 【mysql】 load local data infield 报错 ERROR 1148 (42000): The used command is not allowed with this MySQL version

    mysql> load data local infile '/Users/flint/learn/mysql/pet' into table bx_pet; 执行报错 ERROR 1148 ( ...

  2. php源码安装执行configure报错error: off_t undefined; check your library configuration

    php安装执行configure报错error: off_t undefined; check your library configuration vim /etc/ld.so.conf 添加如下几 ...

  3. zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理

    zepplin0.7.2报错ERROR, exception: null, result: %text java.lang.NullPointerException的处理 问题描述: 使用zeppli ...

  4. open数据库报错ERROR at line 1: ORA-03113: end-of-file on communication channel Process ID: 3880 Session ID: 125 Serial number: 3

    1.今天打开数据时,失败,报错 ERROR at line 1:ORA-03113: end-of-file on communication channelProcess ID: 3880Sessi ...

  5. linux使用wkhtmltopdf报错error while loading shared libraries:

    官网提示 linux需要这些动态库.depends on: zlib, fontconfig, freetype, X11 libs (libX11, libXext, libXrender) 在li ...

  6. 发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store

    发布报错:Error ITMS-90635 - Invalid Mach-O in bundle - submitting to App store 昨晚上传项目到AppStore,报了这个错,纳尼! ...

  7. mysql报错"ERROR 1206 (HY000): The total number of locks exceeds the lock table size"的解决方法

    1. 问题背景         InnoDB是新版MySQL(v5.5及以后)默认的存储引擎,之前版本的默认引擎为MyISAM,因此,低于5.5版本的mysql配置文件.my.cnf中,关于InnoD ...

  8. 运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for reposit 今天给Centos通过rpm - ...

  9. 转：运行yum报错Error: Cannot retrieve metalink for reposit

    http://www.netpc.com.cn/593.html 运行yum报错Error: Cannot retrieve metalink for repository: epel. Please ...

随机推荐

  1. Python学习之warn()函数

    warn()函数位于warnings模块中,用来发出警告,或者忽略它或引发异常. def warn(message, category=None, stacklevel=, source=None) ...

  2. 使用 Bolt 实现 GridView 表格控件

    用 Bolt 实现了一个表格控件: 1. 提供 Insert,Remove,Get,Set 接口,可以为表格增删数据: 2. 通过  ItemClass, ItemSetDataFunc 属性来指定显 ...

  3. django的html模板中获取字典的值

    在django的html模板中获取字典中的值应当直接使用 字典.[key] 的方式 {% for i in lists %} <li id="{{i.id}}" class ...

  4. 浅谈HTTPS协议和SSL、TLS之间的区别与关系

    HTTP可能是我们见到过最多的一个字符串了,应该没有之一,而对于HTTPS到来和趋势,我们又开始看到SSL/TLS,所以对于一般不只做技术的人来说这或许还是一个疑问,那么子凡就趁最近在折腾这方面来给大 ...

  5. DelphiXE8怎么使用调试模式(小米手机2)

    需求:在开发Android程序时,大家一直是使用ShowMessage.其实XE是支持下断点的. 操作:小米手机2: 1.小米手机2用USB线,连到电脑上. 2.小米手机2-设置-关于手机-" ...

  6. spring-common.xml

    <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.sp ...

  7. spring-core依赖jar包

  8. 使用ASM获得JAVA类方法参数名

    在Java1.8之后,可以通过反射API java.lang.reflect.Executable.getParameters来获取到方法参数的元信息,(在使用编译器时加上-parameters参数, ...

  9. 元器件封装标准IPC-7351

    IPC-7351依赖久经考验的数学算法,综合考虑制造.组装和元件容差,从而精确计算焊盘图形.该标准以IPC-SM-782研发概念为基础进一步提高,对每一个元件都建立了三个焊盘图形几何形状,对每一系列元 ...

  10. 微信小程序1 - 扩展app.js

    常规的开发过程中, 每个Page的逻辑中,要使用 util.js   WapRequest.js, 需要在每个页面中书写 require('/utils/WapRequest'); 略繁琐 在app. ...