Hadoop CapacitySchedule配置
下面是Hadoop中CapacitySchedule配置,包含了新建队列和子队列
<configuration> <property>
<name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
<value>0.2</value>
</property> <property>
<name>yarn.scheduler.capacity.maximum-applications</name>
<value>10000</value>
</property> <property>
<name>yarn.scheduler.capacity.node-locality-delay</name>
<value>40</value>
</property> <property>
<name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
<value>false</value>
</property> <property>
<name>yarn.scheduler.capacity.resource-calculator</name>
<value>org.apache.hadoop.yarn.util.resource.DominantResourceCalculator</value>
</property> <property>
<name>yarn.scheduler.capacity.root.accessible-node-labels</name>
<value>*</value>
<description></description>
</property> <property>
<name>yarn.scheduler.capacity.root.acl_administer_queue</name>
<value>*</value>
<description></description>
</property> <property>
<name>yarn.scheduler.capacity.root.capacity</name>
<value>100</value>
</property> <property>
<name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
<value>*</value>
</property> <property>
<name>yarn.scheduler.capacity.root.default.capacity</name>
<value>50</value>
</property> <property>
<name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
<value>70</value>
</property> <property>
<name>yarn.scheduler.capacity.root.default.state</name>
<value>RUNNING</value>
</property> <property>
<name>yarn.scheduler.capacity.root.default.user-limit-factor</name>
<value>1</value>
</property> <property>
<name>yarn.scheduler.capacity.root.queues</name>
<value>default,spark,hadoop</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.acl_administer_queue</name>
<value>*</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.acl_submit_applications</name>
<value>*</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.capacity</name>
<value>30</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.maximum-capacity</name>
<value>70</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.minimum-user-limit-percent</name>
<value>100</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.ordering-policy</name>
<value>fifo</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.state</name>
<value>RUNNING</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.user-limit-factor</name>
<value>1</value>
</property> <property>
<name>yarn.scheduler.capacity.root.hadoop.acl_administer_queue</name>
<value>*</value>
</property> <property>
<name>yarn.scheduler.capacity.root.hadoop.acl_submit_applications</name>
<value>*</value>
</property> <property>
<name>yarn.scheduler.capacity.root.hadoop.capacity</name>
<value>20</value>
</property> <property>
<name>yarn.scheduler.capacity.root.hadoop.maximum-capacity</name>
<value>70</value>
</property> <property>
<name>yarn.scheduler.capacity.root.hadoop.minimum-user-limit-percent</name>
<value>100</value>
</property> <property>
<name>yarn.scheduler.capacity.root.hadoop.ordering-policy</name>
<value>fifo</value>
</property> <property>
<name>yarn.scheduler.capacity.root.hadoop.state</name>
<value>RUNNING</value>
</property> <property>
<name>yarn.scheduler.capacity.root.hadoop.user-limit-factor</name>
<value>1</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.queues</name>
<value>spark1,spark2</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark1.acl_administer_queue</name>
<value>*</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark1.acl_submit_applications</name>
<value>*</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark1.capacity</name>
<value>50</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark1.maximum-capacity</name>
<value>70</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark1.minimum-user-limit-percent</name>
<value>100</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark1.ordering-policy</name>
<value>fifo</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark1.state</name>
<value>RUNNING</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark1.user-limit-factor</name>
<value>1</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark2.acl_administer_queue</name>
<value>*</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark2.acl_submit_applications</name>
<value>*</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark2.capacity</name>
<value>50</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark2.maximum-capacity</name>
<value>70</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark2.minimum-user-limit-percent</name>
<value>100</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark2.ordering-policy</name>
<value>fifo</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark2.state</name>
<value>RUNNING</value>
</property> <property>
<name>yarn.scheduler.capacity.root.spark.spark2.user-limit-factor</name>
<value>1</value>
</property>
</configuration>
Hadoop CapacitySchedule配置的更多相关文章
- CentOS 7 Hadoop安装配置
前言:我使用了两台计算机进行集群的配置,如果是单机的话可能会出现部分问题.首先设置两台计算机的主机名 root 权限打开/etc/host文件 再设置hostname,root权限打开/etc/hos ...
- ubuntu下hadoop环境配置
软件环境: 虚拟机:VMware Workstation 10 操作系统:ubuntu-12.04-desktop-amd64 JAVA版本:jdk-7u55-linux-x64 Hadoop版本:h ...
- hadoop(四):配置参数
hadoop参数配置,主要是配置 core-site.xml,hdfs-site.xml,mapred-site.xml 三个配置文件,core-site.xml是全局配置,hdfs-site.xml ...
- hadoop mapred-queue-acls 配置(转)
hadoop作业提交时可以指定相应的队列,例如:-Dmapred.job.queue.name=queue2通过对mapred-queue-acls.xml和mapred-site.xml配置可以对不 ...
- hadoop安装配置——伪分布模式
1. 安装 这里以安装hadoop-0.20.2为例 先安装java,参考这个 去着下载hadoop 解压 2. 配置 修改环境变量 vim ~/.bashrc export HADOOP_HOME= ...
- Hadoop平台配置总结
hadoop的配置,个人感觉是非常容易出问题.一个原因是要配置的地方多,还有个原因就是集群配置要在几台机器上都配置正确,才能保证配置好hadoop,跑起任务. 经过昨晚加今天上午的折腾,总算成功配好了 ...
- 有关hadoop分布式配置详解
linux配置ssh无密码登录 配置ssh无密码登录,先要安装openssh,如下: yum install openssh-clients 准备两台linux服务器或虚拟机,设置两台linux的ho ...
- CentOS Hadoop安装配置详细
总体思路,准备主从服务器,配置主服务器可以无密码SSH登录从服务器,解压安装JDK,解压安装Hadoop,配置hdfs.mapreduce等主从关系. 1.环境,3台CentOS7,64位,Hadoo ...
- Hadoop的配置过程(虚拟机中的伪分布模式)
1引言 hadoop如今已经成为大数据处理中不可缺少的关键技术,在如今大数据爆炸的时代,hadoop给我们处理海量数据提供了强有力的技术支撑.因此,了解hadoop的原理与应用方法是必要的技术知识. ...
随机推荐
- hbase系列之:独立模式部署hbase
一.概述 在上一篇博文中,我简要介绍了hbase的部分基础概念,如果想初步了解hbase的理论,可以参看上一篇博文 hbase系列之:初识hbase .本博文主要介绍独立模式下部署hbase及hbas ...
- Understanding the Bias-Variance Tradeoff
Understanding the Bias-Variance Tradeoff When we discuss prediction models, prediction errors can be ...
- 算法: 排序: 归并排序(Merge)
http://www.codeproject.com/Articles/805587/Merge-Sort
- soj1036. Crypto Columns
1036. Crypto Columns Constraints Time Limit: 1 secs, Memory Limit: 32 MB Description The columnar en ...
- npm_一个有意思的npm包
$ npm install yosay const yosay = require('yosay'); console.log(yosay('Hello, and welcome to my fant ...
- Chrome 清除某个特定网站下的缓存
打开开发者工具(F12),选择 Network--Disable cache 即可.需要清除某网站缓存时 F12 打开开发者工具就会自动清除这个网站的缓存,而不必清除所有网站的缓存了.
- 查看Linux系统版本的几种方法
第一种: cat /etc/os-release # 或者 cat /etc/redhat-release 结果如下: NAME="Ubuntu" VERSION="16 ...
- equals方法变量和常量位置区别
对于字符串比较,我的习惯用法是 变量.equals(常量) 比如: a.equals("a") 今天看视频才知道变量在前面与后面有很大影响,正确的写法是常量放前面(可以 ...
- 冲量:momentum
参见:http://www.jianshu.com/p/58b3fe300ecb,这个博客里有冲量的python实现的代码和讲解 “冲量”这个概念源自于物理中的力学,表示力对时间的积累效应. 在普通的 ...
- 读书笔记 effective c++ Item 52 如果你实现了placement new,你也要实现placement delete
1. 调用普通版本的operator new抛出异常会发生什么? Placement new和placement delete不是C++动物园中最常遇到的猛兽,所以你不用担心你对它们不熟悉.当你像下面 ...