一、下载安装包

解压安装包并创建data,name,tmp文件夹

二、修改配置文件

1、core-site.xml

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  3. <!--
  4.   Licensed under the Apache License, Version 2.0 (the "License");
  5.   you may not use this file except in compliance with the License.
  6.   You may obtain a copy of the License at
  7.  
  8.     http://www.apache.org/licenses/LICENSE-2.0
  9.  
  10.   Unless required by applicable law or agreed to in writing, software
  11.   distributed under the License is distributed on an "AS IS" BASIS,
  12.   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  13.   See the License for the specific language governing permissions and
  14.   limitations under the License. See accompanying LICENSE file.
  15. -->
  16.  
  17. <!-- Put site-specific property overrides in this file. -->
  18.  
  19. <configuration>
  20.     <property>
  21.         <name>hadoop.tmp.dir</name>
  22.         <value>/D:/hadoop/hadoopbak/tmp</value>
  23.     </property>
  24.     <property>
  25.         <name>dfs.name.dir</name>
  26.         <value>/D:/hadoop/hadoopbak/name</value>
  27.     </property>
  28.     <property>
  29.         <name>fs.default.name</name>
  30.         <value>hdfs://localhost:9000</value>
  31.     </property>
  32. </configuration>

2、mapred-site.xml (修改原来的mapred-site.xml.template)

  1. <?xml version="1.0"?>
  2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  3. <!--
  4.   Licensed under the Apache License, Version 2.0 (the "License");
  5.   you may not use this file except in compliance with the License.
  6.   You may obtain a copy of the License at
  7.  
  8.     http://www.apache.org/licenses/LICENSE-2.0
  9.  
  10.   Unless required by applicable law or agreed to in writing, software
  11.   distributed under the License is distributed on an "AS IS" BASIS,
  12.   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  13.   See the License for the specific language governing permissions and
  14.   limitations under the License. See accompanying LICENSE file.
  15. -->
  16.  
  17. <!-- Put site-specific property overrides in this file. -->
  18.  
  19. <configuration>
  20. <property>
  21.        <name>mapreduce.framework.name</name>
  22.        <value>yarn</value>
  23.     </property>
  24.     <property>
  25.        <name>mapred.job.tracker</name>
  26.        <value>hdfs://localhost:9001</value>
  27.     </property>
  28. </configuration>

3、hdfs-site.xml

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  3. <!--
  4.   Licensed under the Apache License, Version 2.0 (the "License");
  5.   you may not use this file except in compliance with the License.
  6.   You may obtain a copy of the License at
  7.  
  8.     http://www.apache.org/licenses/LICENSE-2.0
  9.  
  10.   Unless required by applicable law or agreed to in writing, software
  11.   distributed under the License is distributed on an "AS IS" BASIS,
  12.   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  13.   See the License for the specific language governing permissions and
  14.   limitations under the License. See accompanying LICENSE file.
  15. -->
  16.  
  17. <!-- Put site-specific property overrides in this file. -->
  18.  
  19. <configuration>
  20. <!-- 这个参数设置为1，因为是单机版hadoop -->
  21.     <property>
  22.         <name>dfs.replication</name>
  23.         <value>1</value>
  24.     </property>
  25.     <property>
  26.         <name>dfs.data.dir</name>
  27.         <value>/D:/hadoop/hadoopbak/data</value>
  28.     </property>
  29. </configuration>

4、yarn-site.xml

  1. <?xml version="1.0"?>
  2. <!--
  3.   Licensed under the Apache License, Version 2.0 (the "License");
  4.   you may not use this file except in compliance with the License.
  5.   You may obtain a copy of the License at
  6.  
  7.     http://www.apache.org/licenses/LICENSE-2.0
  8.  
  9.   Unless required by applicable law or agreed to in writing, software
  10.   distributed under the License is distributed on an "AS IS" BASIS,
  11.   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  12.   See the License for the specific language governing permissions and
  13.   limitations under the License. See accompanying LICENSE file.
  14. -->
  15. <configuration>
  16.  
  17. <!-- Site specific YARN configuration properties -->
  18. <property>
  19.        <name>yarn.nodemanager.aux-services</name>
  20.        <value>mapreduce_shuffle</value>
  21.     </property>
  22.     <property>
  23.        <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
  24.        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  25.     </property>
  26. </configuration>

四、替换原下载安装包中的bin目录

五、运行准备

1、hadoop格式化

hdfs namenode -format

2、切换到sbin目录之下执行start-all.cmd

启动yarn的时候提示错误Couldn't find a package.json file 暂未解决

3、操作HDFS进行上传测试

创建目录，上传数据，查看文件。

六、查看hadoop自带的web控制台GUI

conf文件：

  1. This XML file does not appear to have any style information associated with it. The document tree is shown below.
  2. <configuration>
  3. <property>
  4. <name>yarn.ipc.rpc.class</name>
  5. <value>org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC</value>
  6. <source>yarn-default.xml</source>
  7. </property>
  8. <property>
  9. <name>mapreduce.job.maxtaskfailures.per.tracker</name>
  10. <value>3</value>
  11. <source>mapred-default.xml</source>
  12. </property>
  13. <property>
  14. <name>yarn.client.max-cached-nodemanagers-proxies</name>
  15. <value>0</value>
  16. <source>yarn-default.xml</source>
  17. </property>
  18. <property>
  19. <name>mapreduce.job.speculative.retry-after-speculate</name>
  20. <value>15000</value>
  21. <source>mapred-default.xml</source>
  22. </property>
  23. <property>
  24. <name>ha.health-monitor.connect-retry-interval.ms</name>
  25. <value>1000</value>
  26. <source>core-default.xml</source>
  27. </property>
  28. <property>
  29. <name>
  30. yarn.resourcemanager.work-preserving-recovery.enabled
  31. </name>
  32. <value>true</value>
  33. <source>yarn-default.xml</source>
  34. </property>
  35. <property>
  36. <name>mapreduce.reduce.markreset.buffer.percent</name>
  37. <value>0.0</value>
  38. <source>mapred-default.xml</source>
  39. </property>
  40. <property>
  41. <name>mapreduce.jobhistory.max-age-ms</name>
  42. <value>604800000</value>
  43. <source>mapred-default.xml</source>
  44. </property>
  45. <property>
  46. <name>mapreduce.job.ubertask.enable</name>
  47. <value>false</value>
  48. <source>mapred-default.xml</source>
  49. </property>
  50. <property>
  51. <name>
  52. rpc.engine.org.apache.hadoop.yarn.server.api.ResourceTrackerPB
  53. </name>
  54. <value>org.apache.hadoop.ipc.ProtobufRpcEngine</value>
  55. <source>programatically</source>
  56. </property>
  57. <property>
  58. <name>yarn.nodemanager.log-aggregation.compression-type</name>
  59. <value>none</value>
  60. <source>yarn-default.xml</source>
  61. </property>
  62. <property>
  63. <name>mapreduce.job.complete.cancel.delegation.tokens</name>
  64. <value>true</value>
  65. <source>mapred-default.xml</source>
  66. </property>
  67. <property>
  68. <name>mapreduce.jobhistory.datestring.cache.size</name>
  69. <value>200000</value>
  70. <source>mapred-default.xml</source>
  71. </property>
  72. <property>
  73. <name>
  74. hadoop.security.kms.client.authentication.retry-count
  75. </name>
  76. <value>1</value>
  77. <source>core-default.xml</source>
  78. </property>
  79. <property>
  80. <name>hadoop.ssl.enabled.protocols</name>
  81. <value>TLSv1</value>
  82. <source>core-default.xml</source>
  83. </property>
  84. <property>
  85. <name>yarn.resourcemanager.scheduler.address</name>
  86. <value>DESKTOP-H804TCF:8030</value>
  87. <source>programatically</source>
  88. </property>
  89. <property>
  90. <name>hadoop.http.cross-origin.enabled</name>
  91. <value>false</value>
  92. <source>core-default.xml</source>
  93. </property>
  94. <property>
  95. <name>
  96. yarn.nodemanager.container-executor.os.sched.priority.adjustment
  97. </name>
  98. <value>0</value>
  99. <source>yarn-default.xml</source>
  100. </property>
  101. <property>
  102. <name>yarn.resourcemanager.proxy-user-privileges.enabled</name>
  103. <value>false</value>
  104. <source>yarn-default.xml</source>
  105. </property>
  106. <property>
  107. <name>mapreduce.reduce.shuffle.fetch.retry.enabled</name>
  108. <value>${yarn.nodemanager.recovery.enabled}</value>
  109. <source>mapred-default.xml</source>
  110. </property>
  111. <property>
  112. <name>io.mapfile.bloom.error.rate</name>
  113. <value>0.005</value>
  114. <source>core-default.xml</source>
  115. </property>
  116. <property>
  117. <name>yarn.nodemanager.resourcemanager.minimum.version</name>
  118. <value>NONE</value>
  119. <source>yarn-default.xml</source>
  120. </property>
  121. <property>
  122. <name>
  123. yarn.resourcemanager.nodemanagers.heartbeat-interval-ms
  124. </name>
  125. <value>1000</value>
  126. <source>yarn-default.xml</source>
  127. </property>
  128. <property>
  129. <name>hadoop.http.cross-origin.allowed-headers</name>
  130. <value>X-Requested-With,Content-Type,Accept,Origin</value>
  131. <source>core-default.xml</source>
  132. </property>
  133. <property>
  134. <name>yarn.nodemanager.delete.debug-delay-sec</name>
  135. <value>0</value>
  136. <source>yarn-default.xml</source>
  137. </property>
  138. <property>
  139. <name>yarn.scheduler.maximum-allocation-vcores</name>
  140. <value>4</value>
  141. <source>yarn-default.xml</source>
  142. </property>
  143. <property>
  144. <name>yarn.timeline-service.address</name>
  145. <value>${yarn.timeline-service.hostname}:10200</value>
  146. <source>yarn-default.xml</source>
  147. </property>
  148. <property>
  149. <name>ipc.maximum.response.length</name>
  150. <value>134217728</value>
  151. <source>core-default.xml</source>
  152. </property>
  153. <property>
  154. <name>
  155. yarn.nodemanager.disk-health-checker.min-free-space-per-disk-mb
  156. </name>
  157. <value>0</value>
  158. <source>yarn-default.xml</source>
  159. </property>
  160. <property>
  161. <name>mapreduce.job.hdfs-servers</name>
  162. <value>${fs.defaultFS}</value>
  163. <source>yarn-default.xml</source>
  164. </property>
  165. <property>
  166. <name>mapreduce.task.profile.reduce.params</name>
  167. <value>${mapreduce.task.profile.params}</value>
  168. <source>mapred-default.xml</source>
  169. </property>
  170. <property>
  171. <name>ftp.stream-buffer-size</name>
  172. <value>4096</value>
  173. <source>core-default.xml</source>
  174. </property>
  175. <property>
  176. <name>hadoop.http.cross-origin.allowed-methods</name>
  177. <value>GET,POST,HEAD</value>
  178. <source>core-default.xml</source>
  179. </property>
  180. <property>
  181. <name>fs.s3a.buffer.dir</name>
  182. <value>${hadoop.tmp.dir}/s3a</value>
  183. <source>core-default.xml</source>
  184. </property>
  185. <property>
  186. <name>
  187. yarn.client.application-client-protocol.poll-interval-ms
  188. </name>
  189. <value>200</value>
  190. <source>yarn-default.xml</source>
  191. </property>
  192. <property>
  193. <name>yarn.timeline-service.leveldb-timeline-store.path</name>
  194. <value>${hadoop.tmp.dir}/yarn/timeline</value>
  195. <source>yarn-default.xml</source>
  196. </property>
  197. <property>
  198. <name>mapreduce.job.split.metainfo.maxsize</name>
  199. <value>10000000</value>
  200. <source>mapred-default.xml</source>
  201. </property>
  202. <property>
  203. <name>fs.s3a.fast.upload.buffer</name>
  204. <value>disk</value>
  205. <source>core-default.xml</source>
  206. </property>
  207. <property>
  208. <name>s3native.bytes-per-checksum</name>
  209. <value>512</value>
  210. <source>core-default.xml</source>
  211. </property>
  212. <property>
  213. <name>yarn.client.failover-retries-on-socket-timeouts</name>
  214. <value>0</value>
  215. <source>yarn-default.xml</source>
  216. </property>
  217. <property>
  218. <name>hadoop.security.sensitive-config-keys</name>
  219. <value>
  220. secret$,password$,ssl.keystore.pass$,fs.s3.*[Ss]ecret.?[Kk]ey,fs.azure.account.key.*,dfs.webhdfs.oauth2.[a-z]+.token,hadoop.security.sensitive-config-keys
  221. </value>
  222. <source>core-default.xml</source>
  223. </property>
  224. <property>
  225. <name>yarn.timeline-service.client.retry-interval-ms</name>
  226. <value>1000</value>
  227. <source>yarn-default.xml</source>
  228. </property>
  229. <property>
  230. <name>hadoop.http.authentication.type</name>
  231. <value>simple</value>
  232. <source>programatically</source>
  233. </property>
  234. <property>
  235. <name>mapreduce.local.clientfactory.class.name</name>
  236. <value>org.apache.hadoop.mapred.LocalClientFactory</value>
  237. <source>mapred-default.xml</source>
  238. </property>
  239. <property>
  240. <name>ipc.client.connection.maxidletime</name>
  241. <value>10000</value>
  242. <source>core-default.xml</source>
  243. </property>
  244. <property>
  245. <name>ipc.server.max.connections</name>
  246. <value>0</value>
  247. <source>core-default.xml</source>
  248. </property>
  249. <property>
  250. <name>mapreduce.jobhistory.recovery.store.leveldb.path</name>
  251. <value>${hadoop.tmp.dir}/mapred/history/recoverystore</value>
  252. <source>mapred-default.xml</source>
  253. </property>
  254. <property>
  255. <name>fs.s3a.multipart.purge.age</name>
  256. <value>86400</value>
  257. <source>core-default.xml</source>
  258. </property>
  259. <property>
  260. <name>yarn.timeline-service.client.best-effort</name>
  261. <value>false</value>
  262. <source>yarn-default.xml</source>
  263. </property>
  264. <property>
  265. <name>mapreduce.job.ubertask.maxmaps</name>
  266. <value>9</value>
  267. <source>mapred-default.xml</source>
  268. </property>
  269. <property>
  270. <name>
  271. yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage
  272. </name>
  273. <value>90.0</value>
  274. <source>yarn-default.xml</source>
  275. </property>
  276. <property>
  277. <name>mapreduce.ifile.readahead.bytes</name>
  278. <value>4194304</value>
  279. <source>mapred-default.xml</source>
  280. </property>
  281. <property>
  282. <name>mapreduce.jobhistory.admin.address</name>
  283. <value>0.0.0.0:10033</value>
  284. <source>mapred-default.xml</source>
  285. </property>
  286. <property>
  287. <name>yarn.sharedcache.uploader.server.thread-count</name>
  288. <value>50</value>
  289. <source>yarn-default.xml</source>
  290. </property>
  291. <property>
  292. <name>s3.client-write-packet-size</name>
  293. <value>65536</value>
  294. <source>core-default.xml</source>
  295. </property>
  296. <property>
  297. <name>yarn.app.mapreduce.am.resource.cpu-vcores</name>
  298. <value>1</value>
  299. <source>mapred-default.xml</source>
  300. </property>
  301. <property>
  302. <name>mapreduce.input.lineinputformat.linespermap</name>
  303. <value>1</value>
  304. <source>mapred-default.xml</source>
  305. </property>
  306. <property>
  307. <name>mapreduce.reduce.shuffle.input.buffer.percent</name>
  308. <value>0.70</value>
  309. <source>mapred-default.xml</source>
  310. </property>
  311. <property>
  312. <name>hadoop.http.staticuser.user</name>
  313. <value>dr.who</value>
  314. <source>core-default.xml</source>
  315. </property>
  316. <property>
  317. <name>mapreduce.reduce.maxattempts</name>
  318. <value>4</value>
  319. <source>mapred-default.xml</source>
  320. </property>
  321. <property>
  322. <name>mapreduce.jobhistory.admin.acl</name>
  323. <value>*</value>
  324. <source>mapred-default.xml</source>
  325. </property>
  326. <property>
  327. <name>
  328. hadoop.security.group.mapping.ldap.search.filter.user
  329. </name>
  330. <value>(&(objectClass=user)(sAMAccountName={0}))</value>
  331. <source>core-default.xml</source>
  332. </property>
  333. <property>
  334. <name>hadoop.workaround.non.threadsafe.getpwuid</name>
  335. <value>true</value>
  336. <source>core-default.xml</source>
  337. </property>
  338. <property>
  339. <name>mapreduce.map.maxattempts</name>
  340. <value>4</value>
  341. <source>mapred-default.xml</source>
  342. </property>
  343. <property>
  344. <name>mapreduce.jobhistory.cleaner.interval-ms</name>
  345. <value>86400000</value>
  346. <source>mapred-default.xml</source>
  347. </property>
  348. <property>
  349. <name>
  350. yarn.timeline-service.entity-group-fs-store.active-dir
  351. </name>
  352. <value>/tmp/entity-file-history/active</value>
  353. <source>yarn-default.xml</source>
  354. </property>
  355. <property>
  356. <name>yarn.resourcemanager.zk-retry-interval-ms</name>
  357. <value>1000</value>
  358. <source>yarn-default.xml</source>
  359. </property>
  360. <property>
  361. <name>yarn.is.minicluster</name>
  362. <value>false</value>
  363. <source>yarn-default.xml</source>
  364. </property>
  365. <property>
  366. <name>fs.s3n.block.size</name>
  367. <value>67108864</value>
  368. <source>core-default.xml</source>
  369. </property>
  370. <property>
  371. <name>hadoop.registry.system.acls</name>
  372. <value>sasl:yarn@, sasl:mapred@, sasl:hdfs@</value>
  373. <source>core-default.xml</source>
  374. </property>
  375. <property>
  376. <name>
  377. yarn.nodemanager.node-labels.provider.fetch-timeout-ms
  378. </name>
  379. <value>1200000</value>
  380. <source>yarn-default.xml</source>
  381. </property>
  382. <property>
  383. <name>yarn.sharedcache.store.in-memory.check-period-mins</name>
  384. <value>720</value>
  385. <source>yarn-default.xml</source>
  386. </property>
  387. <property>
  388. <name>fs.s3a.multiobjectdelete.enable</name>
  389. <value>true</value>
  390. <source>core-default.xml</source>
  391. </property>
  392. <property>
  393. <name>mapreduce.map.skip.proc-count.auto-incr</name>
  394. <value>true</value>
  395. <source>mapred-default.xml</source>
  396. </property>
  397. <property>
  398. <name>yarn.nodemanager.vmem-check-enabled</name>
  399. <value>true</value>
  400. <source>yarn-default.xml</source>
  401. </property>
  402. <property>
  403. <name>hadoop.security.authentication</name>
  404. <value>simple</value>
  405. <source>core-default.xml</source>
  406. </property>
  407. <property>
  408. <name>mapreduce.reduce.skip.proc-count.auto-incr</name>
  409. <value>true</value>
  410. <source>mapred-default.xml</source>
  411. </property>
  412. <property>
  413. <name>mapreduce.reduce.cpu.vcores</name>
  414. <value>1</value>
  415. <source>mapred-default.xml</source>
  416. </property>
  417. <property>
  418. <name>net.topology.node.switch.mapping.impl</name>
  419. <value>org.apache.hadoop.net.ScriptBasedMapping</value>
  420. <source>core-default.xml</source>
  421. </property>
  422. <property>
  423. <name>fs.s3.sleepTimeSeconds</name>
  424. <value>10</value>
  425. <source>core-default.xml</source>
  426. </property>
  427. <property>
  428. <name>yarn.timeline-service.ttl-ms</name>
  429. <value>604800000</value>
  430. <source>yarn-default.xml</source>
  431. </property>
  432. <property>
  433. <name>yarn.sharedcache.root-dir</name>
  434. <value>/sharedcache</value>
  435. <source>yarn-default.xml</source>
  436. </property>
  437. <property>
  438. <name>yarn.resourcemanager.keytab</name>
  439. <value>/etc/krb5.keytab</value>
  440. <source>yarn-default.xml</source>
  441. </property>
  442. <property>
  443. <name>
  444. yarn.resourcemanager.container.liveness-monitor.interval-ms
  445. </name>
  446. <value>600000</value>
  447. <source>yarn-default.xml</source>
  448. </property>
  449. <property>
  450. <name>
  451. hadoop.security.group.mapping.ldap.posix.attr.gid.name
  452. </name>
  453. <value>gidNumber</value>
  454. <source>core-default.xml</source>
  455. </property>
  456. <property>
  457. <name>
  458. yarn.app.mapreduce.am.scheduler.heartbeat.interval-ms
  459. </name>
  460. <value>1000</value>
  461. <source>mapred-default.xml</source>
  462. </property>
  463. <property>
  464. <name>
  465. yarn.app.mapreduce.client-am.ipc.max-retries-on-timeouts
  466. </name>
  467. <value>3</value>
  468. <source>mapred-default.xml</source>
  469. </property>
  470. <property>
  471. <name>
  472. yarn.nodemanager.linux-container-executor.cgroups.hierarchy
  473. </name>
  474. <value>/hadoop-yarn</value>
  475. <source>yarn-default.xml</source>
  476. </property>
  477. <property>
  478. <name>s3.bytes-per-checksum</name>
  479. <value>512</value>
  480. <source>core-default.xml</source>
  481. </property>
  482. <property>
  483. <name>hadoop.ssl.require.client.cert</name>
  484. <value>false</value>
  485. <source>core-default.xml</source>
  486. </property>
  487. <property>
  488. <name>mapreduce.output.fileoutputformat.compress</name>
  489. <value>false</value>
  490. <source>mapred-default.xml</source>
  491. </property>
  492. <property>
  493. <name>
  494. yarn.resourcemanager.node-labels.provider.fetch-interval-ms
  495. </name>
  496. <value>1800000</value>
  497. <source>yarn-default.xml</source>
  498. </property>
  499. <property>
  500. <name>
  501. yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled
  502. </name>
  503. <value>true</value>
  504. <source>yarn-default.xml</source>
  505. </property>
  506. <property>
  507. <name>mapred.child.tmp</name>
  508. <value>/cygdrive/d/hadoop/hadoop-2.8.0/tmp</value>
  509. <source>core-site.xml</source>
  510. </property>
  511. <property>
  512. <name>mapreduce.shuffle.max.threads</name>
  513. <value>0</value>
  514. <source>mapred-default.xml</source>
  515. </property>
  516. <property>
  517. <name>
  518. yarn.nodemanager.linux-container-executor.cgroups.delete-timeout-ms
  519. </name>
  520. <value>1000</value>
  521. <source>yarn-default.xml</source>
  522. </property>
  523. <property>
  524. <name>s3native.client-write-packet-size</name>
  525. <value>65536</value>
  526. <source>core-default.xml</source>
  527. </property>
  528. <property>
  529. <name>mapreduce.client.submit.file.replication</name>
  530. <value>10</value>
  531. <source>mapred-default.xml</source>
  532. </property>
  533. <property>
  534. <name>yarn.app.mapreduce.am.job.committer.commit-window</name>
  535. <value>10000</value>
  536. <source>mapred-default.xml</source>
  537. </property>
  538. <property>
  539. <name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name>
  540. <value>250</value>
  541. <source>yarn-default.xml</source>
  542. </property>
  543. <property>
  544. <name>yarn.nodemanager.env-whitelist</name>
  545. <value>
  546. JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME
  547. </value>
  548. <source>yarn-default.xml</source>
  549. </property>
  550. <property>
  551. <name>mapreduce.map.speculative</name>
  552. <value>true</value>
  553. <source>mapred-default.xml</source>
  554. </property>
  555. <property>
  556. <name>mapreduce.job.speculative.slowtaskthreshold</name>
  557. <value>1.0</value>
  558. <source>mapred-default.xml</source>
  559. </property>
  560. <property>
  561. <name>
  562. yarn.nodemanager.linux-container-executor.cgroups.mount
  563. </name>
  564. <value>false</value>
  565. <source>yarn-default.xml</source>
  566. </property>
  567. <property>
  568. <name>mapreduce.jobhistory.http.policy</name>
  569. <value>HTTP_ONLY</value>
  570. <source>mapred-default.xml</source>
  571. </property>
  572. <property>
  573. <name>ipc.client.low-latency</name>
  574. <value>false</value>
  575. <source>core-default.xml</source>
  576. </property>
  577. <property>
  578. <name>fs.s3a.paging.maximum</name>
  579. <value>5000</value>
  580. <source>core-default.xml</source>
  581. </property>
  582. <property>
  583. <name>mapreduce.jvm.system-properties-to-log</name>
  584. <value>
  585. os.name,os.version,java.home,java.runtime.version,java.vendor,java.version,java.vm.name,java.class.path,java.io.tmpdir,user.dir,user.name
  586. </value>
  587. <source>mapred-default.xml</source>
  588. </property>
  589. <property>
  590. <name>hadoop.kerberos.min.seconds.before.relogin</name>
  591. <value>60</value>
  592. <source>core-default.xml</source>
  593. </property>
  594. <property>
  595. <name>yarn.resourcemanager.nodemanager-connect-retries</name>
  596. <value>10</value>
  597. <source>yarn-default.xml</source>
  598. </property>
  599. <property>
  600. <name>fs.s3.buffer.dir</name>
  601. <value>${hadoop.tmp.dir}/s3</value>
  602. <source>core-default.xml</source>
  603. </property>
  604. <property>
  605. <name>io.native.lib.available</name>
  606. <value>true</value>
  607. <source>core-default.xml</source>
  608. </property>
  609. <property>
  610. <name>mapreduce.jobhistory.done-dir</name>
  611. <value>${yarn.app.mapreduce.am.staging-dir}/history/done</value>
  612. <source>mapred-default.xml</source>
  613. </property>
  614. <property>
  615. <name>hadoop.registry.zk.retry.interval.ms</name>
  616. <value>1000</value>
  617. <source>core-default.xml</source>
  618. </property>
  619. <property>
  620. <name>
  621. mapreduce.job.reducer.unconditional-preempt.delay.sec
  622. </name>
  623. <value>300</value>
  624. <source>mapred-default.xml</source>
  625. </property>
  626. <property>
  627. <name>hadoop.ssl.hostname.verifier</name>
  628. <value>DEFAULT</value>
  629. <source>core-default.xml</source>
  630. </property>
  631. <property>
  632. <name>mapreduce.task.timeout</name>
  633. <value>600000</value>
  634. <source>mapred-default.xml</source>
  635. </property>
  636. <property>
  637. <name>
  638. yarn.resourcemanager.configuration.file-system-based-store
  639. </name>
  640. <value>/yarn/conf</value>
  641. <source>yarn-default.xml</source>
  642. </property>
  643. <property>
  644. <name>yarn.nodemanager.disk-health-checker.interval-ms</name>
  645. <value>120000</value>
  646. <source>yarn-default.xml</source>
  647. </property>
  648. <property>
  649. <name>hadoop.security.groups.cache.secs</name>
  650. <value>300</value>
  651. <source>core-default.xml</source>
  652. </property>
  653. <property>
  654. <name>mapreduce.input.fileinputformat.split.minsize</name>
  655. <value>0</value>
  656. <source>mapred-default.xml</source>
  657. </property>
  658. <property>
  659. <name>yarn.minicluster.control-resource-monitoring</name>
  660. <value>false</value>
  661. <source>yarn-default.xml</source>
  662. </property>
  663. <property>
  664. <name>yarn.resourcemanager.fail-fast</name>
  665. <value>${yarn.fail-fast}</value>
  666. <source>yarn-default.xml</source>
  667. </property>
  668. <property>
  669. <name>mapreduce.shuffle.port</name>
  670. <value>13562</value>
  671. <source>mapred-default.xml</source>
  672. </property>
  673. <property>
  674. <name>hadoop.rpc.protection</name>
  675. <value>authentication</value>
  676. <source>core-default.xml</source>
  677. </property>
  678. <property>
  679. <name>yarn.client.failover-proxy-provider</name>
  680. <value>
  681. org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider
  682. </value>
  683. <source>yarn-default.xml</source>
  684. </property>
  685. <property>
  686. <name>yarn.timeline-service.recovery.enabled</name>
  687. <value>false</value>
  688. <source>yarn-default.xml</source>
  689. </property>
  690. <property>
  691. <name>ipc.client.tcpnodelay</name>
  692. <value>true</value>
  693. <source>core-default.xml</source>
  694. </property>
  695. <property>
  696. <name>fs.s3.maxRetries</name>
  697. <value>4</value>
  698. <source>core-default.xml</source>
  699. </property>
  700. <property>
  701. <name>hadoop.http.authentication.kerberos.principal</name>
  702. <value>HTTP/_HOST@LOCALHOST</value>
  703. <source>core-default.xml</source>
  704. </property>
  705. <property>
  706. <name>
  707. hadoop.security.group.mapping.ldap.posix.attr.uid.name
  708. </name>
  709. <value>uidNumber</value>
  710. <source>core-default.xml</source>
  711. </property>
  712. <property>
  713. <name>yarn.resourcemanager.webapp.address</name>
  714. <value>${yarn.resourcemanager.hostname}:8088</value>
  715. <source>yarn-default.xml</source>
  716. </property>
  717. <property>
  718. <name>mapreduce.task.profile.reduces</name>
  719. <value>0-2</value>
  720. <source>mapred-default.xml</source>
  721. </property>
  722. <property>
  723. <name>yarn.timeline-service.client.max-retries</name>
  724. <value>30</value>
  725. <source>yarn-default.xml</source>
  726. </property>
  727. <property>
  728. <name>yarn.resourcemanager.am.max-attempts</name>
  729. <value>2</value>
  730. <source>yarn-default.xml</source>
  731. </property>
  732. <property>
  733. <name>mapreduce.job.end-notification.max.retry.interval</name>
  734. <value>5000</value>
  735. <source>mapred-default.xml</source>
  736. </property>
  737. <property>
  738. <name>ipc.client.connect.retry.interval</name>
  739. <value>1000</value>
  740. <source>core-default.xml</source>
  741. </property>
  742. <property>
  743. <name>fs.s3a.multipart.size</name>
  744. <value>100M</value>
  745. <source>core-default.xml</source>
  746. </property>
  747. <property>
  748. <name>yarn.app.mapreduce.am.command-opts</name>
  749. <value>-Xmx1024m</value>
  750. <source>mapred-default.xml</source>
  751. </property>
  752. <property>
  753. <name>yarn.nodemanager.process-kill-wait.ms</name>
  754. <value>2000</value>
  755. <source>yarn-default.xml</source>
  756. </property>
  757. <property>
  758. <name>yarn.timeline-service.state-store-class</name>
  759. <value>
  760. org.apache.hadoop.yarn.server.timeline.recovery.LeveldbTimelineStateStore
  761. </value>
  762. <source>yarn-default.xml</source>
  763. </property>
  764. <property>
  765. <name>
  766. yarn.timeline-service.client.fd-clean-interval-secs
  767. </name>
  768. <value>60</value>
  769. <source>yarn-default.xml</source>
  770. </property>
  771. <property>
  772. <name>mapreduce.job.speculative.minimum-allowed-tasks</name>
  773. <value>10</value>
  774. <source>mapred-default.xml</source>
  775. </property>
  776. <property>
  777. <name>hadoop.jetty.logs.serve.aliases</name>
  778. <value>true</value>
  779. <source>core-default.xml</source>
  780. </property>
  781. <property>
  782. <name>mapreduce.reduce.shuffle.fetch.retry.timeout-ms</name>
  783. <value>30000</value>
  784. <source>mapred-default.xml</source>
  785. </property>
  786. <property>
  787. <name>fs.du.interval</name>
  788. <value>600000</value>
  789. <source>core-default.xml</source>
  790. </property>
  791. <property>
  792. <name>yarn.sharedcache.admin.address</name>
  793. <value>0.0.0.0:8047</value>
  794. <source>yarn-default.xml</source>
  795. </property>
  796. <property>
  797. <name>
  798. yarn.nodemanager.node-labels.provider.fetch-interval-ms
  799. </name>
  800. <value>600000</value>
  801. <source>yarn-default.xml</source>
  802. </property>
  803. <property>
  804. <name>yarn.acl.reservation-enable</name>
  805. <value>false</value>
  806. <source>yarn-default.xml</source>
  807. </property>
  808. <property>
  809. <name>hadoop.security.random.device.file.path</name>
  810. <value>/dev/urandom</value>
  811. <source>core-default.xml</source>
  812. </property>
  813. <property>
  814. <name>mapreduce.task.merge.progress.records</name>
  815. <value>10000</value>
  816. <source>mapred-default.xml</source>
  817. </property>
  818. <property>
  819. <name>yarn.nodemanager.container-metrics.period-ms</name>
  820. <value>-1</value>
  821. <source>yarn-default.xml</source>
  822. </property>
  823. <property>
  824. <name>hadoop.registry.secure</name>
  825. <value>false</value>
  826. <source>core-default.xml</source>
  827. </property>
  828. <property>
  829. <name>hadoop.ssl.client.conf</name>
  830. <value>ssl-client.xml</value>
  831. <source>core-default.xml</source>
  832. </property>
  833. <property>
  834. <name>mapreduce.job.counters.max</name>
  835. <value>120</value>
  836. <source>mapred-default.xml</source>
  837. </property>
  838. <property>
  839. <name>yarn.nodemanager.localizer.fetch.thread-count</name>
  840. <value>4</value>
  841. <source>yarn-default.xml</source>
  842. </property>
  843. <property>
  844. <name>io.mapfile.bloom.size</name>
  845. <value>1048576</value>
  846. <source>core-default.xml</source>
  847. </property>
  848. <property>
  849. <name>yarn.nodemanager.localizer.client.thread-count</name>
  850. <value>5</value>
  851. <source>yarn-default.xml</source>
  852. </property>
  853. <property>
  854. <name>fs.automatic.close</name>
  855. <value>true</value>
  856. <source>core-default.xml</source>
  857. </property>
  858. <property>
  859. <name>mapreduce.task.profile</name>
  860. <value>false</value>
  861. <source>mapred-default.xml</source>
  862. </property>
  863. <property>
  864. <name>yarn.nodemanager.recovery.compaction-interval-secs</name>
  865. <value>3600</value>
  866. <source>yarn-default.xml</source>
  867. </property>
  868. <property>
  869. <name>mapreduce.task.combine.progress.records</name>
  870. <value>10000</value>
  871. <source>mapred-default.xml</source>
  872. </property>
  873. <property>
  874. <name>mapreduce.shuffle.ssl.file.buffer.size</name>
  875. <value>65536</value>
  876. <source>mapred-default.xml</source>
  877. </property>
  878. <property>
  879. <name>yarn.app.mapreduce.client.job.max-retries</name>
  880. <value>0</value>
  881. <source>mapred-default.xml</source>
  882. </property>
  883. <property>
  884. <name>yarn.app.mapreduce.am.container.log.backups</name>
  885. <value>0</value>
  886. <source>mapred-default.xml</source>
  887. </property>
  888. <property>
  889. <name>yarn.minicluster.fixed.ports</name>
  890. <value>false</value>
  891. <source>yarn-default.xml</source>
  892. </property>
  893. <property>
  894. <name>mapreduce.app-submission.cross-platform</name>
  895. <value>false</value>
  896. <source>mapred-default.xml</source>
  897. </property>
  898. <property>
  899. <name>yarn.timeline-service.ttl-enable</name>
  900. <value>true</value>
  901. <source>yarn-default.xml</source>
  902. </property>
  903. <property>
  904. <name>
  905. yarn.nodemanager.container-monitor.procfs-tree.smaps-based-rss.enabled
  906. </name>
  907. <value>false</value>
  908. <source>yarn-default.xml</source>
  909. </property>
  910. <property>
  911. <name>yarn.nodemanager.keytab</name>
  912. <value>/etc/krb5.keytab</value>
  913. <source>yarn-default.xml</source>
  914. </property>
  915. <property>
  916. <name>yarn.nodemanager.log-aggregation.policy.class</name>
  917. <value>
  918. org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AllContainerLogAggregationPolicy
  919. </value>
  920. <source>yarn-default.xml</source>
  921. </property>
  922. <property>
  923. <name>
  924. yarn.client.application-client-protocol.poll-timeout-ms
  925. </name>
  926. <value>-1</value>
  927. <source>yarn-default.xml</source>
  928. </property>
  929. <property>
  930. <name>yarn.resourcemanager.webapp.ui-actions.enabled</name>
  931. <value>true</value>
  932. <source>yarn-default.xml</source>
  933. </property>
  934. <property>
  935. <name>yarn.sharedcache.client-server.address</name>
  936. <value>0.0.0.0:8045</value>
  937. <source>yarn-default.xml</source>
  938. </property>
  939. <property>
  940. <name>yarn.nodemanager.webapp.cross-origin.enabled</name>
  941. <value>false</value>
  942. <source>yarn-default.xml</source>
  943. </property>
  944. <property>
  945. <name>
  946. yarn.nodemanager.runtime.linux.docker.privileged-containers.allowed
  947. </name>
  948. <value>false</value>
  949. <source>yarn-default.xml</source>
  950. </property>
  951. <property>
  952. <name>hadoop.security.instrumentation.requires.admin</name>
  953. <value>false</value>
  954. <source>core-default.xml</source>
  955. </property>
  956. <property>
  957. <name>io.compression.codec.bzip2.library</name>
  958. <value>system-native</value>
  959. <source>core-default.xml</source>
  960. </property>
  961. <property>
  962. <name>hadoop.ssl.keystores.factory.class</name>
  963. <value>
  964. org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory
  965. </value>
  966. <source>core-default.xml</source>
  967. </property>
  968. <property>
  969. <name>mapreduce.task.exit.timeout</name>
  970. <value>60000</value>
  971. <source>mapred-default.xml</source>
  972. </property>
  973. <property>
  974. <name>fs.ftp.host</name>
  975. <value>0.0.0.0</value>
  976. <source>core-default.xml</source>
  977. </property>
  978. <property>
  979. <name>
  980. yarn.app.mapreduce.am.containerlauncher.threadpool-initial-size
  981. </name>
  982. <value>10</value>
  983. <source>mapred-default.xml</source>
  984. </property>
  985. <property>
  986. <name>s3.blocksize</name>
  987. <value>67108864</value>
  988. <source>core-default.xml</source>
  989. </property>
  990. <property>
  991. <name>s3native.stream-buffer-size</name>
  992. <value>4096</value>
  993. <source>core-default.xml</source>
  994. </property>
  995. <property>
  996. <name>yarn.nodemanager.resource.memory-mb</name>
  997. <value>-1</value>
  998. <source>yarn-default.xml</source>
  999. </property>
  1000. <property>
  1001. <name>mapreduce.task.userlog.limit.kb</name>
  1002. <value>0</value>
  1003. <source>mapred-default.xml</source>
  1004. </property>
  1005. <property>
  1006. <name>
  1007. hadoop.security.crypto.codec.classes.aes.ctr.nopadding
  1008. </name>
  1009. <value>
  1010. org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec, org.apache.hadoop.crypto.JceAesCtrCryptoCodec
  1011. </value>
  1012. <source>core-default.xml</source>
  1013. </property>
  1014. <property>
  1015. <name>mapreduce.reduce.speculative</name>
  1016. <value>true</value>
  1017. <source>mapred-default.xml</source>
  1018. </property>
  1019. <property>
  1020. <name>yarn.node-labels.fs-store.impl.class</name>
  1021. <value>
  1022. org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore
  1023. </value>
  1024. <source>yarn-default.xml</source>
  1025. </property>
  1026. <property>
  1027. <name>hadoop.caller.context.max.size</name>
  1028. <value>128</value>
  1029. <source>core-default.xml</source>
  1030. </property>
  1031. <property>
  1032. <name>yarn.client.failover-retries</name>
  1033. <value>0</value>
  1034. <source>yarn-default.xml</source>
  1035. </property>
  1036. <property>
  1037. <name>yarn.nodemanager.resource.cpu-vcores</name>
  1038. <value>-1</value>
  1039. <source>yarn-default.xml</source>
  1040. </property>
  1041. <property>
  1042. <name>mapreduce.jobhistory.recovery.enable</name>
  1043. <value>false</value>
  1044. <source>mapred-default.xml</source>
  1045. </property>
  1046. <property>
  1047. <name>nfs.exports.allowed.hosts</name>
  1048. <value>* rw</value>
  1049. <source>core-default.xml</source>
  1050. </property>
  1051. <property>
  1052. <name>yarn.sharedcache.checksum.algo.impl</name>
  1053. <value>
  1054. org.apache.hadoop.yarn.sharedcache.ChecksumSHA256Impl
  1055. </value>
  1056. <source>yarn-default.xml</source>
  1057. </property>
  1058. <property>
  1059. <name>mapreduce.reduce.shuffle.memory.limit.percent</name>
  1060. <value>0.25</value>
  1061. <source>mapred-default.xml</source>
  1062. </property>
  1063. <property>
  1064. <name>file.replication</name>
  1065. <value>1</value>
  1066. <source>core-default.xml</source>
  1067. </property>
  1068. <property>
  1069. <name>mapreduce.job.reduce.shuffle.consumer.plugin.class</name>
  1070. <value>org.apache.hadoop.mapreduce.task.reduce.Shuffle</value>
  1071. <source>mapred-default.xml</source>
  1072. </property>
  1073. <property>
  1074. <name>mapreduce.task.local-fs.write-limit.bytes</name>
  1075. <value>-1</value>
  1076. <source>mapred-default.xml</source>
  1077. </property>
  1078. <property>
  1079. <name>yarn.app.mapreduce.am.log.level</name>
  1080. <value>INFO</value>
  1081. <source>mapred-default.xml</source>
  1082. </property>
  1083. <property>
  1084. <name>mapreduce.am.max-attempts</name>
  1085. <value>2</value>
  1086. <source>mapred-default.xml</source>
  1087. </property>
  1088. <property>
  1089. <name>mapreduce.shuffle.connection-keep-alive.timeout</name>
  1090. <value>5</value>
  1091. <source>mapred-default.xml</source>
  1092. </property>
  1093. <property>
  1094. <name>mapreduce.job.reduces</name>
  1095. <value>1</value>
  1096. <source>mapred-default.xml</source>
  1097. </property>
  1098. <property>
  1099. <name>
  1100. hadoop.security.group.mapping.ldap.connection.timeout.ms
  1101. </name>
  1102. <value>60000</value>
  1103. <source>core-default.xml</source>
  1104. </property>
  1105. <property>
  1106. <name>yarn.nodemanager.amrmproxy.client.thread-count</name>
  1107. <value>25</value>
  1108. <source>yarn-default.xml</source>
  1109. </property>
  1110. <property>
  1111. <name>
  1112. yarn.app.mapreduce.am.job.task.listener.thread-count
  1113. </name>
  1114. <value>30</value>
  1115. <source>mapred-default.xml</source>
  1116. </property>
  1117. <property>
  1118. <name>yarn.resourcemanager.store.class</name>
  1119. <value>
  1120. org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
  1121. </value>
  1122. <source>yarn-default.xml</source>
  1123. </property>
  1124. <property>
  1125. <name>s3native.replication</name>
  1126. <value>3</value>
  1127. <source>core-default.xml</source>
  1128. </property>
  1129. <property>
  1130. <name>fs.permissions.umask-mode</name>
  1131. <value>022</value>
  1132. <source>core-default.xml</source>
  1133. </property>
  1134. <property>
  1135. <name>
  1136. yarn.resourcemanager.node-ip-cache.expiry-interval-secs
  1137. </name>
  1138. <value>-1</value>
  1139. <source>yarn-default.xml</source>
  1140. </property>
  1141. <property>
  1142. <name>mapreduce.cluster.local.dir</name>
  1143. <value>${hadoop.tmp.dir}/mapred/local</value>
  1144. <source>mapred-default.xml</source>
  1145. </property>
  1146. <property>
  1147. <name>mapreduce.client.output.filter</name>
  1148. <value>FAILED</value>
  1149. <source>mapred-default.xml</source>
  1150. </property>
  1151. <property>
  1152. <name>yarn.nodemanager.pmem-check-enabled</name>
  1153. <value>true</value>
  1154. <source>yarn-default.xml</source>
  1155. </property>
  1156. <property>
  1157. <name>ftp.replication</name>
  1158. <value>3</value>
  1159. <source>core-default.xml</source>
  1160. </property>
  1161. <property>
  1162. <name>
  1163. hadoop.security.group.mapping.ldap.search.attr.member
  1164. </name>
  1165. <value>member</value>
  1166. <source>core-default.xml</source>
  1167. </property>
  1168. <property>
  1169. <name>fs.s3a.max.total.tasks</name>
  1170. <value>5</value>
  1171. <source>core-default.xml</source>
  1172. </property>
  1173. <property>
  1174. <name>yarn.resourcemanager.fs.state-store.num-retries</name>
  1175. <value>0</value>
  1176. <source>yarn-default.xml</source>
  1177. </property>
  1178. <property>
  1179. <name>yarn.timeline-service.leveldb-state-store.path</name>
  1180. <value>${hadoop.tmp.dir}/yarn/timeline</value>
  1181. <source>yarn-default.xml</source>
  1182. </property>
  1183. <property>
  1184. <name>yarn.resourcemanager.resource-tracker.address</name>
  1185. <value>DESKTOP-H804TCF:8031</value>
  1186. <source>programatically</source>
  1187. </property>
  1188. <property>
  1189. <name>yarn.nodemanager.resource.pcores-vcores-multiplier</name>
  1190. <value>1.0</value>
  1191. <source>yarn-default.xml</source>
  1192. </property>
  1193. <property>
  1194. <name>yarn.resourcemanager.scheduler.monitor.enable</name>
  1195. <value>false</value>
  1196. <source>yarn-default.xml</source>
  1197. </property>
  1198. <property>
  1199. <name>fs.trash.checkpoint.interval</name>
  1200. <value>0</value>
  1201. <source>core-default.xml</source>
  1202. </property>
  1203. <property>
  1204. <name>hadoop.registry.zk.retry.times</name>
  1205. <value>5</value>
  1206. <source>core-default.xml</source>
  1207. </property>
  1208. <property>
  1209. <name>
  1210. yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size
  1211. </name>
  1212. <value>10000</value>
  1213. <source>yarn-default.xml</source>
  1214. </property>
  1215. <property>
  1216. <name>s3.stream-buffer-size</name>
  1217. <value>4096</value>
  1218. <source>core-default.xml</source>
  1219. </property>
  1220. <property>
  1221. <name>fs.s3a.connection.maximum</name>
  1222. <value>15</value>
  1223. <source>core-default.xml</source>
  1224. </property>
  1225. <property>
  1226. <name>hadoop.security.dns.log-slow-lookups.enabled</name>
  1227. <value>false</value>
  1228. <source>core-default.xml</source>
  1229. </property>
  1230. <property>
  1231. <name>file.client-write-packet-size</name>
  1232. <value>65536</value>
  1233. <source>core-default.xml</source>
  1234. </property>
  1235. <property>
  1236. <name>hadoop.shell.missing.defaultFs.warning</name>
  1237. <value>false</value>
  1238. <source>core-default.xml</source>
  1239. </property>
  1240. <property>
  1241. <name>fs.s3a.impl</name>
  1242. <value>org.apache.hadoop.fs.s3a.S3AFileSystem</value>
  1243. <source>core-default.xml</source>
  1244. </property>
  1245. <property>
  1246. <name>
  1247. yarn.nodemanager.windows-container.memory-limit.enabled
  1248. </name>
  1249. <value>false</value>
  1250. <source>yarn-default.xml</source>
  1251. </property>
  1252. <property>
  1253. <name>
  1254. rpc.engine.org.apache.hadoop.yarn.api.ApplicationClientProtocolPB
  1255. </name>
  1256. <value>org.apache.hadoop.ipc.ProtobufRpcEngine</value>
  1257. <source>programatically</source>
  1258. </property>
  1259. <property>
  1260. <name>yarn.nodemanager.remote-app-log-dir</name>
  1261. <value>/tmp/logs</value>
  1262. <source>yarn-default.xml</source>
  1263. </property>
  1264. <property>
  1265. <name>mapreduce.reduce.shuffle.retry-delay.max.ms</name>
  1266. <value>60000</value>
  1267. <source>mapred-default.xml</source>
  1268. </property>
  1269. <property>
  1270. <name>io.map.index.interval</name>
  1271. <value>128</value>
  1272. <source>core-default.xml</source>
  1273. </property>
  1274. <property>
  1275. <name>yarn.nodemanager.container-localizer.java.opts</name>
  1276. <value>-Xmx256m</value>
  1277. <source>yarn-default.xml</source>
  1278. </property>
  1279. <property>
  1280. <name>hadoop.ssl.server.conf</name>
  1281. <value>ssl-server.xml</value>
  1282. <source>core-default.xml</source>
  1283. </property>
  1284. <property>
  1285. <name>hadoop.rpc.socket.factory.class.default</name>
  1286. <value>org.apache.hadoop.net.StandardSocketFactory</value>
  1287. <source>core-default.xml</source>
  1288. </property>
  1289. <property>
  1290. <name>
  1291. yarn.minicluster.yarn.nodemanager.resource.memory-mb
  1292. </name>
  1293. <value>4096</value>
  1294. <source>yarn-default.xml</source>
  1295. </property>
  1296. <property>
  1297. <name>yarn.app.mapreduce.client.max-retries</name>
  1298. <value>3</value>
  1299. <source>mapred-default.xml</source>
  1300. </property>
  1301. <property>
  1302. <name>mapreduce.jobhistory.webapp.https.address</name>
  1303. <value>0.0.0.0:19890</value>
  1304. <source>mapred-default.xml</source>
  1305. </property>
  1306. <property>
  1307. <name>yarn.nodemanager.address</name>
  1308. <value>${yarn.nodemanager.hostname}:0</value>
  1309. <source>yarn-default.xml</source>
  1310. </property>
  1311. <property>
  1312. <name>
  1313. yarn.resourcemanager.max-log-aggregation-diagnostics-in-memory
  1314. </name>
  1315. <value>10</value>
  1316. <source>yarn-default.xml</source>
  1317. </property>
  1318. <property>
  1319. <name>
  1320. ha.failover-controller.graceful-fence.rpc-timeout.ms
  1321. </name>
  1322. <value>5000</value>
  1323. <source>core-default.xml</source>
  1324. </property>
  1325. <property>
  1326. <name>
  1327. yarn.resourcemanager.delayed.delegation-token.removal-interval-ms
  1328. </name>
  1329. <value>30000</value>
  1330. <source>yarn-default.xml</source>
  1331. </property>
  1332. <property>
  1333. <name>yarn.timeline-service.enabled</name>
  1334. <value>false</value>
  1335. <source>yarn-default.xml</source>
  1336. </property>
  1337. <property>
  1338. <name>ipc.maximum.data.length</name>
  1339. <value>67108864</value>
  1340. <source>core-default.xml</source>
  1341. </property>
  1342. <property>
  1343. <name>hadoop.security.group.mapping.providers.combined</name>
  1344. <value>true</value>
  1345. <source>core-default.xml</source>
  1346. </property>
  1347. <property>
  1348. <name>hadoop.security.groups.cache.warn.after.ms</name>
  1349. <value>5000</value>
  1350. <source>core-default.xml</source>
  1351. </property>
  1352. <property>
  1353. <name>
  1354. yarn.resourcemanager.fs.state-store.retry-interval-ms
  1355. </name>
  1356. <value>1000</value>
  1357. <source>yarn-default.xml</source>
  1358. </property>
  1359. <property>
  1360. <name>yarn.resourcemanager.zk-acl</name>
  1361. <value>world:anyone:rwcda</value>
  1362. <source>yarn-default.xml</source>
  1363. </property>
  1364. <property>
  1365. <name>yarn.nodemanager.resource-monitor.interval-ms</name>
  1366. <value>3000</value>
  1367. <source>yarn-default.xml</source>
  1368. </property>
  1369. <property>
  1370. <name>
  1371. yarn.nodemanager.resource.detect-hardware-capabilities
  1372. </name>
  1373. <value>false</value>
  1374. <source>yarn-default.xml</source>
  1375. </property>
  1376. <property>
  1377. <name>yarn.sharedcache.app-checker.class</name>
  1378. <value>
  1379. org.apache.hadoop.yarn.server.sharedcachemanager.RemoteAppChecker
  1380. </value>
  1381. <source>yarn-default.xml</source>
  1382. </property>
  1383. <property>
  1384. <name>
  1385. yarn.timeline-service.entity-group-fs-store.retain-seconds
  1386. </name>
  1387. <value>604800</value>
  1388. <source>yarn-default.xml</source>
  1389. </property>
  1390. <property>
  1391. <name>yarn.nodemanager.webapp.https.address</name>
  1392. <value>0.0.0.0:8044</value>
  1393. <source>yarn-default.xml</source>
  1394. </property>
  1395. <property>
  1396. <name>
  1397. yarn.nodemanager.linux-container-executor.cgroups.delete-delay-ms
  1398. </name>
  1399. <value>20</value>
  1400. <source>yarn-default.xml</source>
  1401. </property>
  1402. <property>
  1403. <name>yarn.nodemanager.amrmproxy.enable</name>
  1404. <value>false</value>
  1405. <source>yarn-default.xml</source>
  1406. </property>
  1407. <property>
  1408. <name>
  1409. yarn.resourcemanager.fs.state-store.retry-policy-spec
  1410. </name>
  1411. <value>2000, 500</value>
  1412. <source>yarn-default.xml</source>
  1413. </property>
  1414. <property>
  1415. <name>fs.s3a.fast.upload</name>
  1416. <value>false</value>
  1417. <source>core-default.xml</source>
  1418. </property>
  1419. <property>
  1420. <name>mapreduce.job.committer.setup.cleanup.needed</name>
  1421. <value>true</value>
  1422. <source>mapred-default.xml</source>
  1423. </property>
  1424. <property>
  1425. <name>mapreduce.job.end-notification.retry.attempts</name>
  1426. <value>0</value>
  1427. <source>mapred-default.xml</source>
  1428. </property>
  1429. <property>
  1430. <name>
  1431. yarn.resourcemanager.state-store.max-completed-applications
  1432. </name>
  1433. <value>${yarn.resourcemanager.max-completed-applications}</value>
  1434. <source>yarn-default.xml</source>
  1435. </property>
  1436. <property>
  1437. <name>mapreduce.map.output.compress</name>
  1438. <value>false</value>
  1439. <source>mapred-default.xml</source>
  1440. </property>
  1441. <property>
  1442. <name>mapreduce.jobhistory.cleaner.enable</name>
  1443. <value>true</value>
  1444. <source>mapred-default.xml</source>
  1445. </property>
  1446. <property>
  1447. <name>mapreduce.job.running.reduce.limit</name>
  1448. <value>0</value>
  1449. <source>mapred-default.xml</source>
  1450. </property>
  1451. <property>
  1452. <name>io.seqfile.local.dir</name>
  1453. <value>${hadoop.tmp.dir}/io/local</value>
  1454. <source>core-default.xml</source>
  1455. </property>
  1456. <property>
  1457. <name>mapreduce.reduce.shuffle.read.timeout</name>
  1458. <value>180000</value>
  1459. <source>mapred-default.xml</source>
  1460. </property>
  1461. <property>
  1462. <name>mapreduce.job.queuename</name>
  1463. <value>default</value>
  1464. <source>mapred-default.xml</source>
  1465. </property>
  1466. <property>
  1467. <name>ipc.client.connect.max.retries</name>
  1468. <value>10</value>
  1469. <source>core-default.xml</source>
  1470. </property>
  1471. <property>
  1472. <name>yarn.app.mapreduce.am.staging-dir</name>
  1473. <value>/tmp/hadoop-yarn/staging</value>
  1474. <source>mapred-default.xml</source>
  1475. </property>
  1476. <property>
  1477. <name>yarn.app.mapreduce.client.job.retry-interval</name>
  1478. <value>2000</value>
  1479. <source>mapred-default.xml</source>
  1480. </property>
  1481. <property>
  1482. <name>
  1483. yarn.nodemanager.linux-container-executor.resources-handler.class
  1484. </name>
  1485. <value>
  1486. org.apache.hadoop.yarn.server.nodemanager.util.DefaultLCEResourcesHandler
  1487. </value>
  1488. <source>yarn-default.xml</source>
  1489. </property>
  1490. <property>
  1491. <name>
  1492. yarn.timeline-service.leveldb-timeline-store.read-cache-size
  1493. </name>
  1494. <value>104857600</value>
  1495. <source>yarn-default.xml</source>
  1496. </property>
  1497. <property>
  1498. <name>io.file.buffer.size</name>
  1499. <value>4096</value>
  1500. <source>core-default.xml</source>
  1501. </property>
  1502. <property>
  1503. <name>yarn.resourcemanager.webapp.cross-origin.enabled</name>
  1504. <value>false</value>
  1505. <source>yarn-default.xml</source>
  1506. </property>
  1507. <property>
  1508. <name>
  1509. yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs
  1510. </name>
  1511. <value>86400</value>
  1512. <source>yarn-default.xml</source>
  1513. </property>
  1514. <property>
  1515. <name>yarn.nodemanager.log.deletion-threads-count</name>
  1516. <value>4</value>
  1517. <source>yarn-default.xml</source>
  1518. </property>
  1519. <property>
  1520. <name>ha.zookeeper.parent-znode</name>
  1521. <value>/hadoop-ha</value>
  1522. <source>core-default.xml</source>
  1523. </property>
  1524. <property>
  1525. <name>tfile.io.chunk.size</name>
  1526. <value>1048576</value>
  1527. <source>core-default.xml</source>
  1528. </property>
  1529. <property>
  1530. <name>
  1531. yarn.resourcemanager.work-preserving-recovery.scheduling-wait-ms
  1532. </name>
  1533. <value>10000</value>
  1534. <source>yarn-default.xml</source>
  1535. </property>
  1536. <property>
  1537. <name>yarn.timeline-service.keytab</name>
  1538. <value>/etc/krb5.keytab</value>
  1539. <source>yarn-default.xml</source>
  1540. </property>
  1541. <property>
  1542. <name>yarn.node-labels.enabled</name>
  1543. <value>false</value>
  1544. <source>yarn-default.xml</source>
  1545. </property>
  1546. <property>
  1547. <name>yarn.acl.enable</name>
  1548. <value>false</value>
  1549. <source>yarn-default.xml</source>
  1550. </property>
  1551. <property>
  1552. <name>
  1553. hadoop.security.group.mapping.ldap.directory.search.timeout
  1554. </name>
  1555. <value>10000</value>
  1556. <source>core-default.xml</source>
  1557. </property>
  1558. <property>
  1559. <name>yarn.timeline-service.version</name>
  1560. <value>1.0f</value>
  1561. <source>yarn-default.xml</source>
  1562. </property>
  1563. <property>
  1564. <name>mapreduce.job.token.tracking.ids.enabled</name>
  1565. <value>false</value>
  1566. <source>mapred-default.xml</source>
  1567. </property>
  1568. <property>
  1569. <name>mapreduce.map.output.compress.codec</name>
  1570. <value>org.apache.hadoop.io.compress.DefaultCodec</value>
  1571. <source>mapred-default.xml</source>
  1572. </property>
  1573. <property>
  1574. <name>yarn.sharedcache.enabled</name>
  1575. <value>false</value>
  1576. <source>yarn-default.xml</source>
  1577. </property>
  1578. <property>
  1579. <name>s3.replication</name>
  1580. <value>3</value>
  1581. <source>core-default.xml</source>
  1582. </property>
  1583. <property>
  1584. <name>hadoop.registry.zk.root</name>
  1585. <value>/registry</value>
  1586. <source>core-default.xml</source>
  1587. </property>
  1588. <property>
  1589. <name>tfile.fs.input.buffer.size</name>
  1590. <value>262144</value>
  1591. <source>core-default.xml</source>
  1592. </property>
  1593. <property>
  1594. <name>yarn.timeline-service.http-authentication.type</name>
  1595. <value>simple</value>
  1596. <source>yarn-default.xml</source>
  1597. </property>
  1598. <property>
  1599. <name>
  1600. ha.failover-controller.graceful-fence.connection.retries
  1601. </name>
  1602. <value>1</value>
  1603. <source>core-default.xml</source>
  1604. </property>
  1605. <property>
  1606. <name>net.topology.script.number.args</name>
  1607. <value>100</value>
  1608. <source>core-default.xml</source>
  1609. </property>
  1610. <property>
  1611. <name>fs.s3n.multipart.uploads.block.size</name>
  1612. <value>67108864</value>
  1613. <source>core-default.xml</source>
  1614. </property>
  1615. <property>
  1616. <name>yarn.sharedcache.admin.thread-count</name>
  1617. <value>1</value>
  1618. <source>yarn-default.xml</source>
  1619. </property>
  1620. <property>
  1621. <name>yarn.nodemanager.recovery.dir</name>
  1622. <value>${hadoop.tmp.dir}/yarn-nm-recovery</value>
  1623. <source>yarn-default.xml</source>
  1624. </property>
  1625. <property>
  1626. <name>hadoop.ssl.enabled</name>
  1627. <value>false</value>
  1628. <source>core-default.xml</source>
  1629. </property>
  1630. <property>
  1631. <name>fs.AbstractFileSystem.ftp.impl</name>
  1632. <value>org.apache.hadoop.fs.ftp.FtpFs</value>
  1633. <source>core-default.xml</source>
  1634. </property>
  1635. <property>
  1636. <name>yarn.timeline-service.handler-thread-count</name>
  1637. <value>10</value>
  1638. <source>yarn-default.xml</source>
  1639. </property>
  1640. <property>
  1641. <name>
  1642. yarn.nodemanager.container-metrics.unregister-delay-ms
  1643. </name>
  1644. <value>10000</value>
  1645. <source>yarn-default.xml</source>
  1646. </property>
  1647. <property>
  1648. <name>hadoop.caller.context.enabled</name>
  1649. <value>false</value>
  1650. <source>core-default.xml</source>
  1651. </property>
  1652. <property>
  1653. <name>mapreduce.jobhistory.recovery.store.class</name>
  1654. <value>
  1655. org.apache.hadoop.mapreduce.v2.hs.HistoryServerFileSystemStateStoreService
  1656. </value>
  1657. <source>mapred-default.xml</source>
  1658. </property>
  1659. <property>
  1660. <name>yarn.nodemanager.log.retain-seconds</name>
  1661. <value>10800</value>
  1662. <source>yarn-default.xml</source>
  1663. </property>
  1664. <property>
  1665. <name>yarn.resourcemanager.admin.address</name>
  1666. <value>DESKTOP-H804TCF:8033</value>
  1667. <source>programatically</source>
  1668. </property>
  1669. <property>
  1670. <name>yarn.resourcemanager.recovery.enabled</name>
  1671. <value>false</value>
  1672. <source>yarn-default.xml</source>
  1673. </property>
  1674. <property>
  1675. <name>
  1676. yarn.resourcemanager.ha.automatic-failover.zk-base-path
  1677. </name>
  1678. <value>/yarn-leader-election</value>
  1679. <source>yarn-default.xml</source>
  1680. </property>
  1681. <property>
  1682. <name>fs.AbstractFileSystem.viewfs.impl</name>
  1683. <value>org.apache.hadoop.fs.viewfs.ViewFs</value>
  1684. <source>core-default.xml</source>
  1685. </property>
  1686. <property>
  1687. <name>fs.AbstractFileSystem.hdfs.impl</name>
  1688. <value>org.apache.hadoop.fs.Hdfs</value>
  1689. <source>core-default.xml</source>
  1690. </property>
  1691. <property>
  1692. <name>yarn.resourcemanager.reservation-system.enable</name>
  1693. <value>false</value>
  1694. <source>yarn-default.xml</source>
  1695. </property>
  1696. <property>
  1697. <name>
  1698. mapreduce.job.speculative.speculative-cap-total-tasks
  1699. </name>
  1700. <value>0.01</value>
  1701. <source>mapred-default.xml</source>
  1702. </property>
  1703. <property>
  1704. <name>
  1705. yarn.timeline-service.generic-application-history.max-applications
  1706. </name>
  1707. <value>10000</value>
  1708. <source>yarn-default.xml</source>
  1709. </property>
  1710. <property>
  1711. <name>yarn.sharedcache.nm.uploader.thread-count</name>
  1712. <value>20</value>
  1713. <source>yarn-default.xml</source>
  1714. </property>
  1715. <property>
  1716. <name>yarn.nodemanager.log-container-debug-info.enabled</name>
  1717. <value>false</value>
  1718. <source>yarn-default.xml</source>
  1719. </property>
  1720. <property>
  1721. <name>fs.AbstractFileSystem.s3a.impl</name>
  1722. <value>org.apache.hadoop.fs.s3a.S3A</value>
  1723. <source>core-default.xml</source>
  1724. </property>
  1725. <property>
  1726. <name>yarn.resourcemanager.max-completed-applications</name>
  1727. <value>10000</value>
  1728. <source>yarn-default.xml</source>
  1729. </property>
  1730. <property>
  1731. <name>yarn.nodemanager.log-dirs</name>
  1732. <value>${yarn.log.dir}/userlogs</value>
  1733. <source>yarn-default.xml</source>
  1734. </property>
  1735. <property>
  1736. <name>
  1737. yarn.resourcemanager.node-removal-untracked.timeout-ms
  1738. </name>
  1739. <value>60000</value>
  1740. <source>yarn-default.xml</source>
  1741. </property>
  1742. <property>
  1743. <name>
  1744. yarn.nodemanager.linux-container-executor.nonsecure-mode.user-pattern
  1745. </name>
  1746. <value>^[_.A-Za-z0-9][-@_.A-Za-z0-9]{0,255}?[$]?$</value>
  1747. <source>yarn-default.xml</source>
  1748. </property>
  1749. <property>
  1750. <name>ftp.blocksize</name>
  1751. <value>67108864</value>
  1752. <source>core-default.xml</source>
  1753. </property>
  1754. <property>
  1755. <name>mapreduce.job.acl-modify-job</name>
  1756. <value></value>
  1757. <source>mapred-default.xml</source>
  1758. </property>
  1759. <property>
  1760. <name>fs.defaultFS</name>
  1761. <value>hdfs://127.0.0.1:9999</value>
  1762. <source>core-site.xml</source>
  1763. </property>
  1764. <property>
  1765. <name>yarn.nodemanager.node-labels.resync-interval-ms</name>
  1766. <value>120000</value>
  1767. <source>yarn-default.xml</source>
  1768. </property>
  1769. <property>
  1770. <name>hadoop.http.filter.initializers</name>
  1771. <value>
  1772. org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer,org.apache.hadoop.http.lib.StaticUserWebFilter
  1773. </value>
  1774. <source>programatically</source>
  1775. </property>
  1776. <property>
  1777. <name>fs.s3n.multipart.copy.block.size</name>
  1778. <value>5368709120</value>
  1779. <source>core-default.xml</source>
  1780. </property>
  1781. <property>
  1782. <name>yarn.resourcemanager.connect.max-wait.ms</name>
  1783. <value>900000</value>
  1784. <source>yarn-default.xml</source>
  1785. </property>
  1786. <property>
  1787. <name>
  1788. yarn.timeline-service.entity-group-fs-store.scan-interval-seconds
  1789. </name>
  1790. <value>60</value>
  1791. <source>yarn-default.xml</source>
  1792. </property>
  1793. <property>
  1794. <name>hadoop.security.group.mapping.ldap.ssl</name>
  1795. <value>false</value>
  1796. <source>core-default.xml</source>
  1797. </property>
  1798. <property>
  1799. <name>yarn.intermediate-data-encryption.enable</name>
  1800. <value>false</value>
  1801. <source>yarn-default.xml</source>
  1802. </property>
  1803. <property>
  1804. <name>yarn.sharedcache.store.class</name>
  1805. <value>
  1806. org.apache.hadoop.yarn.server.sharedcachemanager.store.InMemorySCMStore
  1807. </value>
  1808. <source>yarn-default.xml</source>
  1809. </property>
  1810. <property>
  1811. <name>yarn.fail-fast</name>
  1812. <value>false</value>
  1813. <source>yarn-default.xml</source>
  1814. </property>
  1815. <property>
  1816. <name>yarn.resourcemanager.admin.client.thread-count</name>
  1817. <value>1</value>
  1818. <source>yarn-default.xml</source>
  1819. </property>
  1820. <property>
  1821. <name>
  1822. hadoop.security.kms.client.encrypted.key.cache.size
  1823. </name>
  1824. <value>500</value>
  1825. <source>core-default.xml</source>
  1826. </property>
  1827. <property>
  1828. <name>yarn.app.mapreduce.shuffle.log.separate</name>
  1829. <value>true</value>
  1830. <source>mapred-default.xml</source>
  1831. </property>
  1832. <property>
  1833. <name>ipc.client.kill.max</name>
  1834. <value>10</value>
  1835. <source>core-default.xml</source>
  1836. </property>
  1837. <property>
  1838. <name>
  1839. hadoop.security.group.mapping.ldap.search.filter.group
  1840. </name>
  1841. <value>(objectClass=group)</value>
  1842. <source>core-default.xml</source>
  1843. </property>
  1844. <property>
  1845. <name>fs.AbstractFileSystem.file.impl</name>
  1846. <value>org.apache.hadoop.fs.local.LocalFs</value>
  1847. <source>core-default.xml</source>
  1848. </property>
  1849. <property>
  1850. <name>hadoop.http.authentication.kerberos.keytab</name>
  1851. <value>${user.home}/hadoop.keytab</value>
  1852. <source>core-default.xml</source>
  1853. </property>
  1854. <property>
  1855. <name>yarn.client.nodemanager-connect.max-wait-ms</name>
  1856. <value>180000</value>
  1857. <source>yarn-default.xml</source>
  1858. </property>
  1859. <property>
  1860. <name>mapreduce.job.map.output.collector.class</name>
  1861. <value>org.apache.hadoop.mapred.MapTask$MapOutputBuffer</value>
  1862. <source>mapred-default.xml</source>
  1863. </property>
  1864. <property>
  1865. <name>hadoop.security.uid.cache.secs</name>
  1866. <value>14400</value>
  1867. <source>core-default.xml</source>
  1868. </property>
  1869. <property>
  1870. <name>mapreduce.map.cpu.vcores</name>
  1871. <value>1</value>
  1872. <source>mapred-default.xml</source>
  1873. </property>
  1874. <property>
  1875. <name>yarn.log-aggregation.retain-check-interval-seconds</name>
  1876. <value>-1</value>
  1877. <source>yarn-default.xml</source>
  1878. </property>
  1879. <property>
  1880. <name>mapreduce.map.log.level</name>
  1881. <value>INFO</value>
  1882. <source>mapred-default.xml</source>
  1883. </property>
  1884. <property>
  1885. <name>mapred.child.java.opts</name>
  1886. <value>-Xmx200m</value>
  1887. <source>mapred-default.xml</source>
  1888. </property>
  1889. <property>
  1890. <name>yarn.app.mapreduce.am.hard-kill-timeout-ms</name>
  1891. <value>10000</value>
  1892. <source>mapred-default.xml</source>
  1893. </property>
  1894. <property>
  1895. <name>hadoop.registry.zk.session.timeout.ms</name>
  1896. <value>60000</value>
  1897. <source>core-default.xml</source>
  1898. </property>
  1899. <property>
  1900. <name>mapreduce.job.running.map.limit</name>
  1901. <value>0</value>
  1902. <source>mapred-default.xml</source>
  1903. </property>
  1904. <property>
  1905. <name>
  1906. yarn.sharedcache.store.in-memory.initial-delay-mins
  1907. </name>
  1908. <value>10</value>
  1909. <source>yarn-default.xml</source>
  1910. </property>
  1911. <property>
  1912. <name>
  1913. yarn.timeline-service.entity-group-fs-store.cleaner-interval-seconds
  1914. </name>
  1915. <value>3600</value>
  1916. <source>yarn-default.xml</source>
  1917. </property>
  1918. <property>
  1919. <name>yarn.sharedcache.client-server.thread-count</name>
  1920. <value>50</value>
  1921. <source>yarn-default.xml</source>
  1922. </property>
  1923. <property>
  1924. <name>
  1925. yarn.nodemanager.local-cache.max-files-per-directory
  1926. </name>
  1927. <value>8192</value>
  1928. <source>yarn-default.xml</source>
  1929. </property>
  1930. <property>
  1931. <name>s3native.blocksize</name>
  1932. <value>67108864</value>
  1933. <source>core-default.xml</source>
  1934. </property>
  1935. <property>
  1936. <name>mapreduce.client.completion.pollinterval</name>
  1937. <value>5000</value>
  1938. <source>mapred-default.xml</source>
  1939. </property>
  1940. <property>
  1941. <name>fs.s3a.socket.send.buffer</name>
  1942. <value>8192</value>
  1943. <source>core-default.xml</source>
  1944. </property>
  1945. <property>
  1946. <name>mapreduce.job.maps</name>
  1947. <value>2</value>
  1948. <source>mapred-default.xml</source>
  1949. </property>
  1950. <property>
  1951. <name>fs.AbstractFileSystem.swebhdfs.impl</name>
  1952. <value>org.apache.hadoop.fs.SWebHdfs</value>
  1953. <source>core-default.xml</source>
  1954. </property>
  1955. <property>
  1956. <name>mapreduce.job.acl-view-job</name>
  1957. <value></value>
  1958. <source>mapred-default.xml</source>
  1959. </property>
  1960. <property>
  1961. <name>fs.s3a.readahead.range</name>
  1962. <value>64K</value>
  1963. <source>core-default.xml</source>
  1964. </property>
  1965. <property>
  1966. <name>yarn.resourcemanager.connect.retry-interval.ms</name>
  1967. <value>30000</value>
  1968. <source>yarn-default.xml</source>
  1969. </property>
  1970. <property>
  1971. <name>
  1972. yarn.timeline-service.leveldb-timeline-store.ttl-interval-ms
  1973. </name>
  1974. <value>300000</value>
  1975. <source>yarn-default.xml</source>
  1976. </property>
  1977. <property>
  1978. <name>fs.s3a.multipart.threshold</name>
  1979. <value>2147483647</value>
  1980. <source>core-default.xml</source>
  1981. </property>
  1982. <property>
  1983. <name>mapreduce.shuffle.max.connections</name>
  1984. <value>0</value>
  1985. <source>mapred-default.xml</source>
  1986. </property>
  1987. <property>
  1988. <name>hadoop.shell.safely.delete.limit.num.files</name>
  1989. <value>100</value>
  1990. <source>core-default.xml</source>
  1991. </property>
  1992. <property>
  1993. <name>yarn.log-aggregation-enable</name>
  1994. <value>false</value>
  1995. <source>yarn-default.xml</source>
  1996. </property>
  1997. <property>
  1998. <name>mapreduce.task.io.sort.factor</name>
  1999. <value>10</value>
  2000. <source>mapred-default.xml</source>
  2001. </property>
  2002. <property>
  2003. <name>hadoop.security.dns.log-slow-lookups.threshold.ms</name>
  2004. <value>1000</value>
  2005. <source>core-default.xml</source>
  2006. </property>
  2007. <property>
  2008. <name>ha.health-monitor.sleep-after-disconnect.ms</name>
  2009. <value>1000</value>
  2010. <source>core-default.xml</source>
  2011. </property>
  2012. <property>
  2013. <name>ha.zookeeper.session-timeout.ms</name>
  2014. <value>5000</value>
  2015. <source>core-default.xml</source>
  2016. </property>
  2017. <property>
  2018. <name>
  2019. yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users
  2020. </name>
  2021. <value>true</value>
  2022. <source>yarn-default.xml</source>
  2023. </property>
  2024. <property>
  2025. <name>
  2026. mapreduce.input.fileinputformat.list-status.num-threads
  2027. </name>
  2028. <value>1</value>
  2029. <source>mapred-default.xml</source>
  2030. </property>
  2031. <property>
  2032. <name>io.skip.checksum.errors</name>
  2033. <value>false</value>
  2034. <source>core-default.xml</source>
  2035. </property>
  2036. <property>
  2037. <name>yarn.resourcemanager.scheduler.client.thread-count</name>
  2038. <value>50</value>
  2039. <source>yarn-default.xml</source>
  2040. </property>
  2041. <property>
  2042. <name>
  2043. rpc.engine.org.apache.hadoop.ipc.ProtocolMetaInfoPB
  2044. </name>
  2045. <value>org.apache.hadoop.ipc.ProtobufRpcEngine</value>
  2046. <source>programatically</source>
  2047. </property>
  2048. <property>
  2049. <name>mapreduce.jobhistory.move.thread-count</name>
  2050. <value>3</value>
  2051. <source>mapred-default.xml</source>
  2052. </property>
  2053. <property>
  2054. <name>yarn.resourcemanager.zk-state-store.parent-path</name>
  2055. <value>/rmstore</value>
  2056. <source>yarn-default.xml</source>
  2057. </property>
  2058. <property>
  2059. <name>yarn.timeline-service.client.fd-retain-secs</name>
  2060. <value>300</value>
  2061. <source>yarn-default.xml</source>
  2062. </property>
  2063. <property>
  2064. <name>ipc.client.idlethreshold</name>
  2065. <value>4000</value>
  2066. <source>core-default.xml</source>
  2067. </property>
  2068. <property>
  2069. <name>yarn.sharedcache.cleaner.initial-delay-mins</name>
  2070. <value>10</value>
  2071. <source>yarn-default.xml</source>
  2072. </property>
  2073. <property>
  2074. <name>mapreduce.task.profile.params</name>
  2075. <value>
  2076. -agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,verbose=n,file=%s
  2077. </value>
  2078. <source>mapred-default.xml</source>
  2079. </property>
  2080. <property>
  2081. <name>mapreduce.jobhistory.keytab</name>
  2082. <value>/etc/security/keytab/jhs.service.keytab</value>
  2083. <source>mapred-default.xml</source>
  2084. </property>
  2085. <property>
  2086. <name>yarn.scheduler.minimum-allocation-mb</name>
  2087. <value>1024</value>
  2088. <source>yarn-default.xml</source>
  2089. </property>
  2090. <property>
  2091. <name>
  2092. yarn.resourcemanager.container-tokens.master-key-rolling-interval-secs
  2093. </name>
  2094. <value>86400</value>
  2095. <source>yarn-default.xml</source>
  2096. </property>
  2097. <property>
  2098. <name>mapreduce.reduce.shuffle.fetch.retry.interval-ms</name>
  2099. <value>1000</value>
  2100. <source>mapred-default.xml</source>
  2101. </property>
  2102. <property>
  2103. <name>
  2104. yarn.timeline-service.entity-group-fs-store.app-cache-size
  2105. </name>
  2106. <value>10</value>
  2107. <source>yarn-default.xml</source>
  2108. </property>
  2109. <property>
  2110. <name>hadoop.user.group.static.mapping.overrides</name>
  2111. <value>dr.who=;</value>
  2112. <source>core-default.xml</source>
  2113. </property>
  2114. <property>
  2115. <name>
  2116. hadoop.security.kms.client.encrypted.key.cache.low-watermark
  2117. </name>
  2118. <value>0.3f</value>
  2119. <source>core-default.xml</source>
  2120. </property>
  2121. <property>
  2122. <name>yarn.dispatcher.exit-on-error</name>
  2123. <value>true</value>
  2124. <source>programatically</source>
  2125. </property>
  2126. <property>
  2127. <name>fs.s3a.connection.ssl.enabled</name>
  2128. <value>true</value>
  2129. <source>core-default.xml</source>
  2130. </property>
  2131. <property>
  2132. <name>yarn.nodemanager.runtime.linux.docker.capabilities</name>
  2133. <value>
  2134. CHOWN,DAC_OVERRIDE,FSETID,FOWNER,MKNOD,NET_RAW,SETGID,SETUID,SETFCAP,SETPCAP,NET_BIND_SERVICE,SYS_CHROOT,KILL,AUDIT_WRITE
  2135. </value>
  2136. <source>yarn-default.xml</source>
  2137. </property>
  2138. <property>
  2139. <name>yarn.node-labels.fs-store.retry-policy-spec</name>
  2140. <value>2000, 500</value>
  2141. <source>yarn-default.xml</source>
  2142. </property>
  2143. <property>
  2144. <name>fs.AbstractFileSystem.webhdfs.impl</name>
  2145. <value>org.apache.hadoop.fs.WebHdfs</value>
  2146. <source>core-default.xml</source>
  2147. </property>
  2148. <property>
  2149. <name>yarn.resourcemanager.scheduler.monitor.policies</name>
  2150. <value>
  2151. org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy
  2152. </value>
  2153. <source>yarn-default.xml</source>
  2154. </property>
  2155. <property>
  2156. <name>ipc.server.listen.queue.size</name>
  2157. <value>128</value>
  2158. <source>core-default.xml</source>
  2159. </property>
  2160. <property>
  2161. <name>rpc.metrics.quantile.enable</name>
  2162. <value>false</value>
  2163. <source>core-default.xml</source>
  2164. </property>
  2165. <property>
  2166. <name>
  2167. yarn.nodemanager.resource.system-reserved-memory-mb
  2168. </name>
  2169. <value>-1</value>
  2170. <source>yarn-default.xml</source>
  2171. </property>
  2172. <property>
  2173. <name>
  2174. yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds
  2175. </name>
  2176. <value>-1</value>
  2177. <source>yarn-default.xml</source>
  2178. </property>
  2179. <property>
  2180. <name>
  2181. yarn.client.nodemanager-client-async.thread-pool-max-size
  2182. </name>
  2183. <value>500</value>
  2184. <source>yarn-default.xml</source>
  2185. </property>
  2186. <property>
  2187. <name>hadoop.security.group.mapping</name>
  2188. <value>
  2189. org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback
  2190. </value>
  2191. <source>core-default.xml</source>
  2192. </property>
  2193. <property>
  2194. <name>
  2195. yarn.resourcemanager.system-metrics-publisher.enabled
  2196. </name>
  2197. <value>false</value>
  2198. <source>yarn-default.xml</source>
  2199. </property>
  2200. <property>
  2201. <name>yarn.am.liveness-monitor.expiry-interval-ms</name>
  2202. <value>600000</value>
  2203. <source>yarn-default.xml</source>
  2204. </property>
  2205. <property>
  2206. <name>yarn.nm.liveness-monitor.expiry-interval-ms</name>
  2207. <value>600000</value>
  2208. <source>yarn-default.xml</source>
  2209. </property>
  2210. <property>
  2211. <name>ftp.bytes-per-checksum</name>
  2212. <value>512</value>
  2213. <source>core-default.xml</source>
  2214. </property>
  2215. <property>
  2216. <name>yarn.sharedcache.nested-level</name>
  2217. <value>3</value>
  2218. <source>yarn-default.xml</source>
  2219. </property>
  2220. <property>
  2221. <name>
  2222. rpc.engine.org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocolPB
  2223. </name>
  2224. <value>org.apache.hadoop.ipc.ProtobufRpcEngine</value>
  2225. <source>programatically</source>
  2226. </property>
  2227. <property>
  2228. <name>mapreduce.job.emit-timeline-data</name>
  2229. <value>false</value>
  2230. <source>mapred-default.xml</source>
  2231. </property>
  2232. <property>
  2233. <name>mapreduce.map.memory.mb</name>
  2234. <value>1024</value>
  2235. <source>mapred-default.xml</source>
  2236. </property>
  2237. <property>
  2238. <name>yarn.client.nodemanager-connect.retry-interval-ms</name>
  2239. <value>10000</value>
  2240. <source>yarn-default.xml</source>
  2241. </property>
  2242. <property>
  2243. <name>hadoop.http.cross-origin.max-age</name>
  2244. <value>1800</value>
  2245. <source>core-default.xml</source>
  2246. </property>
  2247. <property>
  2248. <name>
  2249. yarn.timeline-service.leveldb-timeline-store.start-time-read-cache-size
  2250. </name>
  2251. <value>10000</value>
  2252. <source>yarn-default.xml</source>
  2253. </property>
  2254. <property>
  2255. <name>yarn.scheduler.include-port-in-node-name</name>
  2256. <value>false</value>
  2257. <source>yarn-default.xml</source>
  2258. </property>
  2259. <property>
  2260. <name>mapreduce.job.speculative.retry-after-no-speculate</name>
  2261. <value>1000</value>
  2262. <source>mapred-default.xml</source>
  2263. </property>
  2264. <property>
  2265. <name>hadoop.registry.zk.connection.timeout.ms</name>
  2266. <value>15000</value>
  2267. <source>core-default.xml</source>
  2268. </property>
  2269. <property>
  2270. <name>yarn.resourcemanager.address</name>
  2271. <value>DESKTOP-H804TCF:8032</value>
  2272. <source>programatically</source>
  2273. </property>
  2274. <property>
  2275. <name>ipc.client.rpc-timeout.ms</name>
  2276. <value>0</value>
  2277. <source>core-default.xml</source>
  2278. </property>
  2279. <property>
  2280. <name>mapreduce.task.skip.start.attempts</name>
  2281. <value>2</value>
  2282. <source>mapred-default.xml</source>
  2283. </property>
  2284. <property>
  2285. <name>fs.s3a.socket.recv.buffer</name>
  2286. <value>8192</value>
  2287. <source>core-default.xml</source>
  2288. </property>
  2289. <property>
  2290. <name>yarn.resourcemanager.zk-timeout-ms</name>
  2291. <value>10000</value>
  2292. <source>yarn-default.xml</source>
  2293. </property>
  2294. <property>
  2295. <name>
  2296. yarn.timeline-service.entity-group-fs-store.summary-store
  2297. </name>
  2298. <value>
  2299. org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore
  2300. </value>
  2301. <source>yarn-default.xml</source>
  2302. </property>
  2303. <property>
  2304. <name>
  2305. hadoop.security.groups.cache.background.reload.threads
  2306. </name>
  2307. <value>3</value>
  2308. <source>core-default.xml</source>
  2309. </property>
  2310. <property>
  2311. <name>yarn.sharedcache.cleaner.resource-sleep-ms</name>
  2312. <value>0</value>
  2313. <source>yarn-default.xml</source>
  2314. </property>
  2315. <property>
  2316. <name>mapreduce.map.skip.maxrecords</name>
  2317. <value>0</value>
  2318. <source>mapred-default.xml</source>
  2319. </property>
  2320. <property>
  2321. <name>
  2322. yarn.resourcemanager.system-metrics-publisher.dispatcher.pool-size
  2323. </name>
  2324. <value>10</value>
  2325. <source>yarn-default.xml</source>
  2326. </property>
  2327. <property>
  2328. <name>yarn.timeline-service.hostname</name>
  2329. <value>0.0.0.0</value>
  2330. <source>yarn-default.xml</source>
  2331. </property>
  2332. <property>
  2333. <name>hadoop.registry.rm.enabled</name>
  2334. <value>false</value>
  2335. <source>core-default.xml</source>
  2336. </property>
  2337. <property>
  2338. <name>mapreduce.job.reducer.preempt.delay.sec</name>
  2339. <value>0</value>
  2340. <source>mapred-default.xml</source>
  2341. </property>
  2342. <property>
  2343. <name>yarn.node-labels.configuration-type</name>
  2344. <value>centralized</value>
  2345. <source>yarn-default.xml</source>
  2346. </property>
  2347. <property>
  2348. <name>mapreduce.shuffle.ssl.enabled</name>
  2349. <value>false</value>
  2350. <source>mapred-default.xml</source>
  2351. </property>
  2352. <property>
  2353. <name>yarn.nodemanager.vmem-pmem-ratio</name>
  2354. <value>2.1</value>
  2355. <source>yarn-default.xml</source>
  2356. </property>
  2357. <property>
  2358. <name>yarn.nodemanager.container-manager.thread-count</name>
  2359. <value>20</value>
  2360. <source>yarn-default.xml</source>
  2361. </property>
  2362. <property>
  2363. <name>hadoop.tmp.dir</name>
  2364. <value>/tmp/hadoop-${user.name}</value>
  2365. <source>core-default.xml</source>
  2366. </property>
  2367. <property>
  2368. <name>fs.AbstractFileSystem.har.impl</name>
  2369. <value>org.apache.hadoop.fs.HarFs</value>
  2370. <source>core-default.xml</source>
  2371. </property>
  2372. <property>
  2373. <name>yarn.nodemanager.localizer.cache.target-size-mb</name>
  2374. <value>10240</value>
  2375. <source>yarn-default.xml</source>
  2376. </property>
  2377. <property>
  2378. <name>yarn.app.mapreduce.shuffle.log.backups</name>
  2379. <value>0</value>
  2380. <source>mapred-default.xml</source>
  2381. </property>
  2382. <property>
  2383. <name>yarn.minicluster.use-rpc</name>
  2384. <value>false</value>
  2385. <source>yarn-default.xml</source>
  2386. </property>
  2387. <property>
  2388. <name>yarn.http.policy</name>
  2389. <value>HTTP_ONLY</value>
  2390. <source>yarn-default.xml</source>
  2391. </property>
  2392. <property>
  2393. <name>yarn.timeline-service.webapp.https.address</name>
  2394. <value>${yarn.timeline-service.hostname}:8190</value>
  2395. <source>yarn-default.xml</source>
  2396. </property>
  2397. <property>
  2398. <name>yarn.resourcemanager.amlauncher.thread-count</name>
  2399. <value>50</value>
  2400. <source>yarn-default.xml</source>
  2401. </property>
  2402. <property>
  2403. <name>tfile.fs.output.buffer.size</name>
  2404. <value>262144</value>
  2405. <source>core-default.xml</source>
  2406. </property>
  2407. <property>
  2408. <name>fs.ftp.host.port</name>
  2409. <value>21</value>
  2410. <source>core-default.xml</source>
  2411. </property>
  2412. <property>
  2413. <name>mapreduce.task.io.sort.mb</name>
  2414. <value>100</value>
  2415. <source>mapred-default.xml</source>
  2416. </property>
  2417. <property>
  2418. <name>
  2419. hadoop.security.group.mapping.ldap.search.attr.group.name
  2420. </name>
  2421. <value>cn</value>
  2422. <source>core-default.xml</source>
  2423. </property>
  2424. <property>
  2425. <name>yarn.nodemanager.amrmproxy.address</name>
  2426. <value>0.0.0.0:8048</value>
  2427. <source>yarn-default.xml</source>
  2428. </property>
  2429. <property>
  2430. <name>hadoop.security.group.mapping.ldap.read.timeout.ms</name>
  2431. <value>60000</value>
  2432. <source>core-default.xml</source>
  2433. </property>
  2434. <property>
  2435. <name>mapreduce.output.fileoutputformat.compress.type</name>
  2436. <value>RECORD</value>
  2437. <source>mapred-default.xml</source>
  2438. </property>
  2439. <property>
  2440. <name>file.bytes-per-checksum</name>
  2441. <value>512</value>
  2442. <source>core-default.xml</source>
  2443. </property>
  2444. <property>
  2445. <name>ha.health-monitor.check-interval.ms</name>
  2446. <value>1000</value>
  2447. <source>core-default.xml</source>
  2448. </property>
  2449. <property>
  2450. <name>
  2451. yarn.resourcemanager.delegation.key.update-interval
  2452. </name>
  2453. <value>86400000</value>
  2454. <source>yarn-default.xml</source>
  2455. </property>
  2456. <property>
  2457. <name>
  2458. yarn.resourcemanager.resource-tracker.client.thread-count
  2459. </name>
  2460. <value>50</value>
  2461. <source>yarn-default.xml</source>
  2462. </property>
  2463. <property>
  2464. <name>mapreduce.reduce.input.buffer.percent</name>
  2465. <value>0.0</value>
  2466. <source>mapred-default.xml</source>
  2467. </property>
  2468. <property>
  2469. <name>
  2470. yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage
  2471. </name>
  2472. <value>false</value>
  2473. <source>yarn-default.xml</source>
  2474. </property>
  2475. <property>
  2476. <name>ha.health-monitor.rpc-timeout.ms</name>
  2477. <value>45000</value>
  2478. <source>core-default.xml</source>
  2479. </property>
  2480. <property>
  2481. <name>io.bytes.per.checksum</name>
  2482. <value>512</value>
  2483. <source>core-default.xml</source>
  2484. </property>
  2485. <property>
  2486. <name>yarn.scheduler.maximum-allocation-mb</name>
  2487. <value>8192</value>
  2488. <source>yarn-default.xml</source>
  2489. </property>
  2490. <property>
  2491. <name>yarn.resourcemanager.leveldb-state-store.path</name>
  2492. <value>${hadoop.tmp.dir}/yarn/system/rmstore</value>
  2493. <source>yarn-default.xml</source>
  2494. </property>
  2495. <property>
  2496. <name>mapreduce.task.files.preserve.failedtasks</name>
  2497. <value>false</value>
  2498. <source>mapred-default.xml</source>
  2499. </property>
  2500. <property>
  2501. <name>yarn.nodemanager.delete.thread-count</name>
  2502. <value>4</value>
  2503. <source>yarn-default.xml</source>
  2504. </property>
  2505. <property>
  2506. <name>mapreduce.output.fileoutputformat.compress.codec</name>
  2507. <value>org.apache.hadoop.io.compress.DefaultCodec</value>
  2508. <source>mapred-default.xml</source>
  2509. </property>
  2510. <property>
  2511. <name>map.sort.class</name>
  2512. <value>org.apache.hadoop.util.QuickSort</value>
  2513. <source>mapred-default.xml</source>
  2514. </property>
  2515. <property>
  2516. <name>
  2517. yarn.nodemanager.resource.count-logical-processors-as-cores
  2518. </name>
  2519. <value>false</value>
  2520. <source>yarn-default.xml</source>
  2521. </property>
  2522. <property>
  2523. <name>mapreduce.jobhistory.jobname.limit</name>
  2524. <value>50</value>
  2525. <source>mapred-default.xml</source>
  2526. </property>
  2527. <property>
  2528. <name>mapreduce.job.classloader</name>
  2529. <value>false</value>
  2530. <source>mapred-default.xml</source>
  2531. </property>
  2532. <property>
  2533. <name>hadoop.registry.zk.retry.ceiling.ms</name>
  2534. <value>60000</value>
  2535. <source>core-default.xml</source>
  2536. </property>
  2537. <property>
  2538. <name>io.seqfile.compress.blocksize</name>
  2539. <value>1000000</value>
  2540. <source>core-default.xml</source>
  2541. </property>
  2542. <property>
  2543. <name>mapreduce.task.profile.maps</name>
  2544. <value>0-2</value>
  2545. <source>mapred-default.xml</source>
  2546. </property>
  2547. <property>
  2548. <name>
  2549. yarn.nodemanager.localizer.cache.cleanup.interval-ms
  2550. </name>
  2551. <value>600000</value>
  2552. <source>yarn-default.xml</source>
  2553. </property>
  2554. <property>
  2555. <name>hadoop.http.cross-origin.allowed-origins</name>
  2556. <value>*</value>
  2557. <source>core-default.xml</source>
  2558. </property>
  2559. <property>
  2560. <name>
  2561. yarn.timeline-service.client.fd-flush-interval-secs
  2562. </name>
  2563. <value>10</value>
  2564. <source>yarn-default.xml</source>
  2565. </property>
  2566. <property>
  2567. <name>hadoop.security.java.secure.random.algorithm</name>
  2568. <value>SHA1PRNG</value>
  2569. <source>core-default.xml</source>
  2570. </property>
  2571. <property>
  2572. <name>fs.client.resolve.remote.symlinks</name>
  2573. <value>true</value>
  2574. <source>core-default.xml</source>
  2575. </property>
  2576. <property>
  2577. <name>
  2578. yarn.resourcemanager.delegation-token-renewer.thread-count
  2579. </name>
  2580. <value>50</value>
  2581. <source>yarn-default.xml</source>
  2582. </property>
  2583. <property>
  2584. <name>mapreduce.shuffle.listen.queue.size</name>
  2585. <value>128</value>
  2586. <source>mapred-default.xml</source>
  2587. </property>
  2588. <property>
  2589. <name>
  2590. yarn.nodemanager.disk-health-checker.min-healthy-disks
  2591. </name>
  2592. <value>0.25</value>
  2593. <source>yarn-default.xml</source>
  2594. </property>
  2595. <property>
  2596. <name>mapreduce.job.end-notification.retry.interval</name>
  2597. <value>1000</value>
  2598. <source>mapred-default.xml</source>
  2599. </property>
  2600. <property>
  2601. <name>mapreduce.jobhistory.loadedjobs.cache.size</name>
  2602. <value>5</value>
  2603. <source>mapred-default.xml</source>
  2604. </property>
  2605. <property>
  2606. <name>fs.s3a.fast.upload.active.blocks</name>
  2607. <value>4</value>
  2608. <source>core-default.xml</source>
  2609. </property>
  2610. <property>
  2611. <name>yarn.nodemanager.local-dirs</name>
  2612. <value>${hadoop.tmp.dir}/nm-local-dir</value>
  2613. <source>yarn-default.xml</source>
  2614. </property>
  2615. <property>
  2616. <name>mapreduce.task.exit.timeout.check-interval-ms</name>
  2617. <value>20000</value>
  2618. <source>mapred-default.xml</source>
  2619. </property>
  2620. <property>
  2621. <name>hadoop.registry.jaas.context</name>
  2622. <value>Client</value>
  2623. <source>core-default.xml</source>
  2624. </property>
  2625. <property>
  2626. <name>yarn.timeline-service.webapp.address</name>
  2627. <value>${yarn.timeline-service.hostname}:8188</value>
  2628. <source>yarn-default.xml</source>
  2629. </property>
  2630. <property>
  2631. <name>mapreduce.jobhistory.address</name>
  2632. <value>0.0.0.0:10020</value>
  2633. <source>mapred-default.xml</source>
  2634. </property>
  2635. <property>
  2636. <name>ipc.server.log.slow.rpc</name>
  2637. <value>false</value>
  2638. <source>core-default.xml</source>
  2639. </property>
  2640. <property>
  2641. <name>file.blocksize</name>
  2642. <value>67108864</value>
  2643. <source>core-default.xml</source>
  2644. </property>
  2645. <property>
  2646. <name>fs.s3a.block.size</name>
  2647. <value>32M</value>
  2648. <source>core-default.xml</source>
  2649. </property>
  2650. <property>
  2651. <name>
  2652. yarn.timeline-service.entity-group-fs-store.leveldb-cache-read-cache-size
  2653. </name>
  2654. <value>10485760</value>
  2655. <source>yarn-default.xml</source>
  2656. </property>
  2657. <property>
  2658. <name>yarn.sharedcache.cleaner.period-mins</name>
  2659. <value>1440</value>
  2660. <source>yarn-default.xml</source>
  2661. </property>
  2662. <property>
  2663. <name>yarn.resourcemanager.metrics.runtime.buckets</name>
  2664. <value>60,300,1440</value>
  2665. <source>yarn-default.xml</source>
  2666. </property>
  2667. <property>
  2668. <name>ipc.client.ping</name>
  2669. <value>true</value>
  2670. <source>core-default.xml</source>
  2671. </property>
  2672. <property>
  2673. <name>
  2674. yarn.resourcemanager.leveldb-state-store.compaction-interval-secs
  2675. </name>
  2676. <value>3600</value>
  2677. <source>yarn-default.xml</source>
  2678. </property>
  2679. <property>
  2680. <name>yarn.resourcemanager.configuration.provider-class</name>
  2681. <value>org.apache.hadoop.yarn.LocalConfigurationProvider</value>
  2682. <source>yarn-default.xml</source>
  2683. </property>
  2684. <property>
  2685. <name>yarn.nodemanager.recovery.enabled</name>
  2686. <value>false</value>
  2687. <source>yarn-default.xml</source>
  2688. </property>
  2689. <property>
  2690. <name>yarn.resourcemanager.hostname</name>
  2691. <value>0.0.0.0</value>
  2692. <source>yarn-default.xml</source>
  2693. </property>
  2694. <property>
  2695. <name>fs.s3n.multipart.uploads.enabled</name>
  2696. <value>false</value>
  2697. <source>core-default.xml</source>
  2698. </property>
  2699. <property>
  2700. <name>yarn.nodemanager.disk-health-checker.enable</name>
  2701. <value>true</value>
  2702. <source>yarn-default.xml</source>
  2703. </property>
  2704. <property>
  2705. <name>
  2706. yarn.nodemanager.amrmproxy.interceptor-class.pipeline
  2707. </name>
  2708. <value>
  2709. org.apache.hadoop.yarn.server.nodemanager.amrmproxy.DefaultRequestInterceptor
  2710. </value>
  2711. <source>yarn-default.xml</source>
  2712. </property>
  2713. <property>
  2714. <name>ha.failover-controller.cli-check.rpc-timeout.ms</name>
  2715. <value>20000</value>
  2716. <source>core-default.xml</source>
  2717. </property>
  2718. <property>
  2719. <name>ftp.client-write-packet-size</name>
  2720. <value>65536</value>
  2721. <source>core-default.xml</source>
  2722. </property>
  2723. <property>
  2724. <name>mapreduce.reduce.shuffle.parallelcopies</name>
  2725. <value>5</value>
  2726. <source>mapred-default.xml</source>
  2727. </property>
  2728. <property>
  2729. <name>hadoop.caller.context.signature.max.size</name>
  2730. <value>40</value>
  2731. <source>core-default.xml</source>
  2732. </property>
  2733. <property>
  2734. <name>mapreduce.jobhistory.principal</name>
  2735. <value>jhs/_HOST@REALM.TLD</value>
  2736. <source>mapred-default.xml</source>
  2737. </property>
  2738. <property>
  2739. <name>
  2740. hadoop.http.authentication.simple.anonymous.allowed
  2741. </name>
  2742. <value>true</value>
  2743. <source>core-default.xml</source>
  2744. </property>
  2745. <property>
  2746. <name>yarn.log-aggregation.retain-seconds</name>
  2747. <value>-1</value>
  2748. <source>yarn-default.xml</source>
  2749. </property>
  2750. <property>
  2751. <name>
  2752. yarn.resourcemanager.rm.container-allocation.expiry-interval-ms
  2753. </name>
  2754. <value>600000</value>
  2755. <source>yarn-default.xml</source>
  2756. </property>
  2757. <property>
  2758. <name>
  2759. yarn.nodemanager.windows-container.cpu-limit.enabled
  2760. </name>
  2761. <value>false</value>
  2762. <source>yarn-default.xml</source>
  2763. </property>
  2764. <property>
  2765. <name>mapreduce.client.genericoptionsparser.used</name>
  2766. <value>true</value>
  2767. <source>programatically</source>
  2768. </property>
  2769. <property>
  2770. <name>
  2771. yarn.timeline-service.http-authentication.simple.anonymous.allowed
  2772. </name>
  2773. <value>true</value>
  2774. <source>yarn-default.xml</source>
  2775. </property>
  2776. <property>
  2777. <name>mapreduce.jobhistory.jhist.format</name>
  2778. <value>json</value>
  2779. <source>mapred-default.xml</source>
  2780. </property>
  2781. <property>
  2782. <name>
  2783. yarn.resourcemanager.reservation-system.planfollower.time-step
  2784. </name>
  2785. <value>1000</value>
  2786. <source>yarn-default.xml</source>
  2787. </property>
  2788. <property>
  2789. <name>mapreduce.job.ubertask.maxreduces</name>
  2790. <value>1</value>
  2791. <source>mapred-default.xml</source>
  2792. </property>
  2793. <property>
  2794. <name>fs.s3a.connection.establish.timeout</name>
  2795. <value>5000</value>
  2796. <source>core-default.xml</source>
  2797. </property>
  2798. <property>
  2799. <name>yarn.nodemanager.health-checker.interval-ms</name>
  2800. <value>600000</value>
  2801. <source>yarn-default.xml</source>
  2802. </property>
  2803. <property>
  2804. <name>fs.s3a.multipart.purge</name>
  2805. <value>false</value>
  2806. <source>core-default.xml</source>
  2807. </property>
  2808. <property>
  2809. <name>
  2810. hadoop.security.kms.client.encrypted.key.cache.num.refill.threads
  2811. </name>
  2812. <value>2</value>
  2813. <source>core-default.xml</source>
  2814. </property>
  2815. <property>
  2816. <name>yarn.timeline-service.store-class</name>
  2817. <value>
  2818. org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore
  2819. </value>
  2820. <source>yarn-default.xml</source>
  2821. </property>
  2822. <property>
  2823. <name>mapreduce.shuffle.transfer.buffer.size</name>
  2824. <value>131072</value>
  2825. <source>mapred-default.xml</source>
  2826. </property>
  2827. <property>
  2828. <name>yarn.resourcemanager.zk-num-retries</name>
  2829. <value>1000</value>
  2830. <source>yarn-default.xml</source>
  2831. </property>
  2832. <property>
  2833. <name>
  2834. yarn.sharedcache.store.in-memory.staleness-period-mins
  2835. </name>
  2836. <value>10080</value>
  2837. <source>yarn-default.xml</source>
  2838. </property>
  2839. <property>
  2840. <name>yarn.nodemanager.webapp.address</name>
  2841. <value>${yarn.nodemanager.hostname}:8042</value>
  2842. <source>yarn-default.xml</source>
  2843. </property>
  2844. <property>
  2845. <name>yarn.app.mapreduce.client-am.ipc.max-retries</name>
  2846. <value>3</value>
  2847. <source>mapred-default.xml</source>
  2848. </property>
  2849. <property>
  2850. <name>ipc.ping.interval</name>
  2851. <value>60000</value>
  2852. <source>core-default.xml</source>
  2853. </property>
  2854. <property>
  2855. <name>ha.failover-controller.new-active.rpc-timeout.ms</name>
  2856. <value>60000</value>
  2857. <source>core-default.xml</source>
  2858. </property>
  2859. <property>
  2860. <name>mapreduce.jobhistory.client.thread-count</name>
  2861. <value>10</value>
  2862. <source>mapred-default.xml</source>
  2863. </property>
  2864. <property>
  2865. <name>fs.trash.interval</name>
  2866. <value>0</value>
  2867. <source>core-default.xml</source>
  2868. </property>
  2869. <property>
  2870. <name>mapreduce.fileoutputcommitter.algorithm.version</name>
  2871. <value>1</value>
  2872. <source>mapred-default.xml</source>
  2873. </property>
  2874. <property>
  2875. <name>mapreduce.reduce.skip.maxgroups</name>
  2876. <value>0</value>
  2877. <source>mapred-default.xml</source>
  2878. </property>
  2879. <property>
  2880. <name>mapreduce.reduce.memory.mb</name>
  2881. <value>1024</value>
  2882. <source>mapred-default.xml</source>
  2883. </property>
  2884. <property>
  2885. <name>yarn.nodemanager.health-checker.script.timeout-ms</name>
  2886. <value>1200000</value>
  2887. <source>yarn-default.xml</source>
  2888. </property>
  2889. <property>
  2890. <name>mapreduce.client.progressmonitor.pollinterval</name>
  2891. <value>1000</value>
  2892. <source>mapred-default.xml</source>
  2893. </property>
  2894. <property>
  2895. <name>
  2896. yarn.resourcemanager.delegation.token.renew-interval
  2897. </name>
  2898. <value>86400000</value>
  2899. <source>yarn-default.xml</source>
  2900. </property>
  2901. <property>
  2902. <name>yarn.nodemanager.hostname</name>
  2903. <value>0.0.0.0</value>
  2904. <source>yarn-default.xml</source>
  2905. </property>
  2906. <property>
  2907. <name>yarn.resourcemanager.ha.enabled</name>
  2908. <value>false</value>
  2909. <source>yarn-default.xml</source>
  2910. </property>
  2911. <property>
  2912. <name>yarn.scheduler.minimum-allocation-vcores</name>
  2913. <value>1</value>
  2914. <source>yarn-default.xml</source>
  2915. </property>
  2916. <property>
  2917. <name>yarn.app.mapreduce.am.container.log.limit.kb</name>
  2918. <value>0</value>
  2919. <source>mapred-default.xml</source>
  2920. </property>
  2921. <property>
  2922. <name>mapreduce.jobhistory.move.interval-ms</name>
  2923. <value>180000</value>
  2924. <source>mapred-default.xml</source>
  2925. </property>
  2926. <property>
  2927. <name>hadoop.http.authentication.signature.secret.file</name>
  2928. <value>${user.home}/hadoop-http-auth-signature-secret</value>
  2929. <source>core-default.xml</source>
  2930. </property>
  2931. <property>
  2932. <name>
  2933. yarn.resourcemanager.nm-tokens.master-key-rolling-interval-secs
  2934. </name>
  2935. <value>86400</value>
  2936. <source>yarn-default.xml</source>
  2937. </property>
  2938. <property>
  2939. <name>yarn.nodemanager.container-executor.class</name>
  2940. <value>
  2941. org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor
  2942. </value>
  2943. <source>yarn-default.xml</source>
  2944. </property>
  2945. <property>
  2946. <name>hadoop.security.authorization</name>
  2947. <value>false</value>
  2948. <source>core-default.xml</source>
  2949. </property>
  2950. <property>
  2951. <name>yarn.nodemanager.localizer.address</name>
  2952. <value>${yarn.nodemanager.hostname}:8040</value>
  2953. <source>yarn-default.xml</source>
  2954. </property>
  2955. <property>
  2956. <name>mapreduce.jobhistory.recovery.store.fs.uri</name>
  2957. <value>${hadoop.tmp.dir}/mapred/history/recoverystore</value>
  2958. <source>mapred-default.xml</source>
  2959. </property>
  2960. <property>
  2961. <name>mapreduce.shuffle.connection-keep-alive.enable</name>
  2962. <value>false</value>
  2963. <source>mapred-default.xml</source>
  2964. </property>
  2965. <property>
  2966. <name>hadoop.common.configuration.version</name>
  2967. <value>0.23.0</value>
  2968. <source>core-default.xml</source>
  2969. </property>
  2970. <property>
  2971. <name>yarn.app.mapreduce.task.container.log.backups</name>
  2972. <value>0</value>
  2973. <source>mapred-default.xml</source>
  2974. </property>
  2975. <property>
  2976. <name>hadoop.security.groups.negative-cache.secs</name>
  2977. <value>30</value>
  2978. <source>core-default.xml</source>
  2979. </property>
  2980. <property>
  2981. <name>mapreduce.ifile.readahead</name>
  2982. <value>true</value>
  2983. <source>mapred-default.xml</source>
  2984. </property>
  2985. <property>
  2986. <name>
  2987. yarn.nodemanager.resource.percentage-physical-cpu-limit
  2988. </name>
  2989. <value>100</value>
  2990. <source>yarn-default.xml</source>
  2991. </property>
  2992. <property>
  2993. <name>mapreduce.job.max.split.locations</name>
  2994. <value>10</value>
  2995. <source>mapred-default.xml</source>
  2996. </property>
  2997. <property>
  2998. <name>hadoop.registry.zk.quorum</name>
  2999. <value>localhost:2181</value>
  3000. <source>core-default.xml</source>
  3001. </property>
  3002. <property>
  3003. <name>fs.s3a.threads.keepalivetime</name>
  3004. <value>60</value>
  3005. <source>core-default.xml</source>
  3006. </property>
  3007. <property>
  3008. <name>mapreduce.jobhistory.joblist.cache.size</name>
  3009. <value>20000</value>
  3010. <source>mapred-default.xml</source>
  3011. </property>
  3012. <property>
  3013. <name>mapreduce.job.end-notification.max.attempts</name>
  3014. <value>5</value>
  3015. <source>mapred-default.xml</source>
  3016. </property>
  3017. <property>
  3018. <name>hadoop.security.groups.cache.background.reload</name>
  3019. <value>false</value>
  3020. <source>core-default.xml</source>
  3021. </property>
  3022. <property>
  3023. <name>mapreduce.reduce.shuffle.connect.timeout</name>
  3024. <value>180000</value>
  3025. <source>mapred-default.xml</source>
  3026. </property>
  3027. <property>
  3028. <name>mapreduce.jobhistory.webapp.address</name>
  3029. <value>0.0.0.0:19888</value>
  3030. <source>mapred-default.xml</source>
  3031. </property>
  3032. <property>
  3033. <name>fs.s3a.connection.timeout</name>
  3034. <value>200000</value>
  3035. <source>core-default.xml</source>
  3036. </property>
  3037. <property>
  3038. <name>yarn.sharedcache.nm.uploader.replication.factor</name>
  3039. <value>10</value>
  3040. <source>yarn-default.xml</source>
  3041. </property>
  3042. <property>
  3043. <name>hadoop.http.authentication.token.validity</name>
  3044. <value>36000</value>
  3045. <source>core-default.xml</source>
  3046. </property>
  3047. <property>
  3048. <name>ipc.client.connect.max.retries.on.timeouts</name>
  3049. <value>45</value>
  3050. <source>core-default.xml</source>
  3051. </property>
  3052. <property>
  3053. <name>
  3054. yarn.timeline-service.client.internal-timers-ttl-secs
  3055. </name>
  3056. <value>420</value>
  3057. <source>yarn-default.xml</source>
  3058. </property>
  3059. <property>
  3060. <name>
  3061. yarn.nodemanager.docker-container-executor.exec-name
  3062. </name>
  3063. <value>/usr/bin/docker</value>
  3064. <source>yarn-default.xml</source>
  3065. </property>
  3066. <property>
  3067. <name>yarn.app.mapreduce.am.job.committer.cancel-timeout</name>
  3068. <value>60000</value>
  3069. <source>mapred-default.xml</source>
  3070. </property>
  3071. <property>
  3072. <name>dfs.ha.fencing.ssh.connect-timeout</name>
  3073. <value>30000</value>
  3074. <source>core-default.xml</source>
  3075. </property>
  3076. <property>
  3077. <name>mapreduce.reduce.log.level</name>
  3078. <value>INFO</value>
  3079. <source>mapred-default.xml</source>
  3080. </property>
  3081. <property>
  3082. <name>mapreduce.reduce.shuffle.merge.percent</name>
  3083. <value>0.66</value>
  3084. <source>mapred-default.xml</source>
  3085. </property>
  3086. <property>
  3087. <name>ipc.client.fallback-to-simple-auth-allowed</name>
  3088. <value>false</value>
  3089. <source>core-default.xml</source>
  3090. </property>
  3091. <property>
  3092. <name>io.serializations</name>
  3093. <value>
  3094. org.apache.hadoop.io.serializer.WritableSerialization, org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization, org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
  3095. </value>
  3096. <source>core-default.xml</source>
  3097. </property>
  3098. <property>
  3099. <name>fs.s3.block.size</name>
  3100. <value>67108864</value>
  3101. <source>core-default.xml</source>
  3102. </property>
  3103. <property>
  3104. <name>
  3105. yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user
  3106. </name>
  3107. <value>nobody</value>
  3108. <source>yarn-default.xml</source>
  3109. </property>
  3110. <property>
  3111. <name>hadoop.kerberos.kinit.command</name>
  3112. <value>kinit</value>
  3113. <source>core-default.xml</source>
  3114. </property>
  3115. <property>
  3116. <name>
  3117. hadoop.security.kms.client.encrypted.key.cache.expiry
  3118. </name>
  3119. <value>43200000</value>
  3120. <source>core-default.xml</source>
  3121. </property>
  3122. <property>
  3123. <name>yarn.resourcemanager.fs.state-store.uri</name>
  3124. <value>${hadoop.tmp.dir}/yarn/system/rmstore</value>
  3125. <source>yarn-default.xml</source>
  3126. </property>
  3127. <property>
  3128. <name>yarn.dispatcher.drain-events.timeout</name>
  3129. <value>300000</value>
  3130. <source>yarn-default.xml</source>
  3131. </property>
  3132. <property>
  3133. <name>yarn.admin.acl</name>
  3134. <value>*</value>
  3135. <source>yarn-default.xml</source>
  3136. </property>
  3137. <property>
  3138. <name>mapreduce.reduce.merge.inmem.threshold</name>
  3139. <value>1000</value>
  3140. <source>mapred-default.xml</source>
  3141. </property>
  3142. <property>
  3143. <name>yarn.cluster.max-application-priority</name>
  3144. <value>0</value>
  3145. <source>yarn-default.xml</source>
  3146. </property>
  3147. <property>
  3148. <name>net.topology.impl</name>
  3149. <value>org.apache.hadoop.net.NetworkTopology</value>
  3150. <source>core-default.xml</source>
  3151. </property>
  3152. <property>
  3153. <name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
  3154. <value>true</value>
  3155. <source>yarn-default.xml</source>
  3156. </property>
  3157. <property>
  3158. <name>yarn.resourcemanager.scheduler.class</name>
  3159. <value>
  3160. org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
  3161. </value>
  3162. <source>yarn-default.xml</source>
  3163. </property>
  3164. <property>
  3165. <name>io.map.index.skip</name>
  3166. <value>0</value>
  3167. <source>core-default.xml</source>
  3168. </property>
  3169. <property>
  3170. <name>yarn.resourcemanager.webapp.https.address</name>
  3171. <value>${yarn.resourcemanager.hostname}:8090</value>
  3172. <source>yarn-default.xml</source>
  3173. </property>
  3174. <property>
  3175. <name>yarn.nodemanager.admin-env</name>
  3176. <value>MALLOC_ARENA_MAX=$MALLOC_ARENA_MAX</value>
  3177. <source>yarn-default.xml</source>
  3178. </property>
  3179. <property>
  3180. <name>hadoop.security.crypto.cipher.suite</name>
  3181. <value>AES/CTR/NoPadding</value>
  3182. <source>core-default.xml</source>
  3183. </property>
  3184. <property>
  3185. <name>mapreduce.task.profile.map.params</name>
  3186. <value>${mapreduce.task.profile.params}</value>
  3187. <source>mapred-default.xml</source>
  3188. </property>
  3189. <property>
  3190. <name>hadoop.security.crypto.buffer.size</name>
  3191. <value>8192</value>
  3192. <source>core-default.xml</source>
  3193. </property>
  3194. <property>
  3195. <name>
  3196. yarn.nodemanager.aux-services.mapreduce_shuffle.class
  3197. </name>
  3198. <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  3199. <source>yarn-default.xml</source>
  3200. </property>
  3201. <property>
  3202. <name>yarn.nodemanager.container-metrics.enable</name>
  3203. <value>true</value>
  3204. <source>yarn-default.xml</source>
  3205. </property>
  3206. <property>
  3207. <name>fs.s3a.path.style.access</name>
  3208. <value>false</value>
  3209. <source>core-default.xml</source>
  3210. </property>
  3211. <property>
  3212. <name>mapreduce.cluster.acls.enabled</name>
  3213. <value>false</value>
  3214. <source>mapred-default.xml</source>
  3215. </property>
  3216. <property>
  3217. <name>yarn.sharedcache.uploader.server.address</name>
  3218. <value>0.0.0.0:8046</value>
  3219. <source>yarn-default.xml</source>
  3220. </property>
  3221. <property>
  3222. <name>yarn.log-aggregation-status.time-out.ms</name>
  3223. <value>600000</value>
  3224. <source>yarn-default.xml</source>
  3225. </property>
  3226. <property>
  3227. <name>fs.s3a.threads.max</name>
  3228. <value>10</value>
  3229. <source>core-default.xml</source>
  3230. </property>
  3231. <property>
  3232. <name>fs.har.impl.disable.cache</name>
  3233. <value>true</value>
  3234. <source>core-default.xml</source>
  3235. </property>
  3236. <property>
  3237. <name>ipc.client.connect.timeout</name>
  3238. <value>20000</value>
  3239. <source>core-default.xml</source>
  3240. </property>
  3241. <property>
  3242. <name>yarn.nodemanager.remote-app-log-dir-suffix</name>
  3243. <value>logs</value>
  3244. <source>yarn-default.xml</source>
  3245. </property>
  3246. <property>
  3247. <name>fs.df.interval</name>
  3248. <value>60000</value>
  3249. <source>core-default.xml</source>
  3250. </property>
  3251. <property>
  3252. <name>hadoop.util.hash.type</name>
  3253. <value>murmur</value>
  3254. <source>core-default.xml</source>
  3255. </property>
  3256. <property>
  3257. <name>mapreduce.jobhistory.minicluster.fixed.ports</name>
  3258. <value>false</value>
  3259. <source>mapred-default.xml</source>
  3260. </property>
  3261. <property>
  3262. <name>yarn.app.mapreduce.shuffle.log.limit.kb</name>
  3263. <value>0</value>
  3264. <source>mapred-default.xml</source>
  3265. </property>
  3266. <property>
  3267. <name>ha.zookeeper.acl</name>
  3268. <value>world:anyone:rwcda</value>
  3269. <source>core-default.xml</source>
  3270. </property>
  3271. <property>
  3272. <name>
  3273. yarn.timeline-service.entity-group-fs-store.done-dir
  3274. </name>
  3275. <value>/tmp/entity-file-history/done/</value>
  3276. <source>yarn-default.xml</source>
  3277. </property>
  3278. <property>
  3279. <name>yarn.resourcemanager.delegation.token.max-lifetime</name>
  3280. <value>604800000</value>
  3281. <source>yarn-default.xml</source>
  3282. </property>
  3283. <property>
  3284. <name>
  3285. mapreduce.job.speculative.speculative-cap-running-tasks
  3286. </name>
  3287. <value>0.1</value>
  3288. <source>mapred-default.xml</source>
  3289. </property>
  3290. <property>
  3291. <name>mapreduce.map.sort.spill.percent</name>
  3292. <value>0.80</value>
  3293. <source>mapred-default.xml</source>
  3294. </property>
  3295. <property>
  3296. <name>yarn.nodemanager.recovery.supervised</name>
  3297. <value>false</value>
  3298. <source>yarn-default.xml</source>
  3299. </property>
  3300. <property>
  3301. <name>file.stream-buffer-size</name>
  3302. <value>4096</value>
  3303. <source>core-default.xml</source>
  3304. </property>
  3305. <property>
  3306. <name>
  3307. yarn.resourcemanager.ha.automatic-failover.embedded
  3308. </name>
  3309. <value>true</value>
  3310. <source>yarn-default.xml</source>
  3311. </property>
  3312. <property>
  3313. <name>yarn.resourcemanager.nodemanager.minimum.version</name>
  3314. <value>NONE</value>
  3315. <source>yarn-default.xml</source>
  3316. </property>
  3317. <property>
  3318. <name>
  3319. yarn.resourcemanager.history-writer.multi-threaded-dispatcher.pool-size
  3320. </name>
  3321. <value>10</value>
  3322. <source>yarn-default.xml</source>
  3323. </property>
  3324. <property>
  3325. <name>yarn.sharedcache.webapp.address</name>
  3326. <value>0.0.0.0:8788</value>
  3327. <source>yarn-default.xml</source>
  3328. </property>
  3329. <property>
  3330. <name>yarn.app.mapreduce.am.resource.mb</name>
  3331. <value>1536</value>
  3332. <source>mapred-default.xml</source>
  3333. </property>
  3334. <property>
  3335. <name>mapreduce.framework.name</name>
  3336. <value>local</value>
  3337. <source>mapred-default.xml</source>
  3338. </property>
  3339. <property>
  3340. <name>mapreduce.job.reduce.slowstart.completedmaps</name>
  3341. <value>0.05</value>
  3342. <source>mapred-default.xml</source>
  3343. </property>
  3344. <property>
  3345. <name>yarn.resourcemanager.client.thread-count</name>
  3346. <value>50</value>
  3347. <source>yarn-default.xml</source>
  3348. </property>
  3349. <property>
  3350. <name>mapreduce.jobhistory.intermediate-done-dir</name>
  3351. <value>
  3352. ${yarn.app.mapreduce.am.staging-dir}/history/done_intermediate
  3353. </value>
  3354. <source>mapred-default.xml</source>
  3355. </property>
  3356. <property>
  3357. <name>fs.s3a.attempts.maximum</name>
  3358. <value>20</value>
  3359. <source>core-default.xml</source>
  3360. </property>
  3361. </configuration>

【大数据系列】win10不借助Cygwin安装hadoop2.8的更多相关文章

  1. 【大数据系列】hadoop单机模式安装

    一.添加用户和用户组 adduser hadoop 将hadoop用户添加进sudo用户组 sudo usermod -G sudo hadoop 或者 visudo 二.安装jdk 具体操作参考:c ...

  2. 【大数据系列】hadoop单节点安装官方文档翻译

    Hadoop: Setting up a Single Node Cluster. HADOOP:建立单节点集群 Purpose Prerequisites Supported Platforms R ...

  3. 大数据系列之kafka监控kafkaoffsetmonitor安装

    1.下载kafkaoffsetmonitor的jar包,可以到github搜索kafkaoffsetmonitor,第一个就是,里面可以下载编译好了的包. KafkaOffsetMonitor-ass ...

  4. 12.Linux软件安装 (一步一步学习大数据系列之 Linux)

    1.如何上传安装包到服务器 有三种方式: 1.1使用图形化工具,如: filezilla 如何使用FileZilla上传和下载文件 1.2使用 sftp 工具: 在 windows下使用CRT 软件 ...

  5. 大数据系列（5）——Hadoop集群MYSQL的安装

    前言 有一段时间没写文章了,最近事情挺多的,现在咱们回归正题,经过前面四篇文章的介绍,已经通过VMware安装了Hadoop的集群环境,相关的两款软件VSFTP和SecureCRT也已经正常安装了. ...

  6. 大数据系列（4）——Hadoop集群VSFTP和SecureCRT安装配置

    前言 经过前三篇文章的介绍,已经通过VMware安装了Hadoop的集群环境,当然,我相信安装的过程肯定遇到或多或少的问题,这些都需要自己解决,解决的过程就是学习的过程,本篇的来介绍几个Hadoop环 ...

  7. 大数据系列（2）——Hadoop集群坏境CentOS安装

    前言 前面我们主要分析了搭建Hadoop集群所需要准备的内容和一些提前规划好的项,本篇我们主要来分析如何安装CentOS操作系统,以及一些基础的设置,闲言少叙,我们进入本篇的正题. 技术准备 VMwa ...

  8. 大数据系列之数据仓库Hive安装

    Hive系列博文,持续更新~~~ 大数据系列之数据仓库Hive原理 大数据系列之数据仓库Hive安装 大数据系列之数据仓库Hive中分区Partition如何使用 大数据系列之数据仓库Hive命令使用 ...

  9. 大数据系列之分布式数据库HBase-1.2.4+Zookeeper 安装及增删改查实践

    之前介绍过关于HBase 0.9.8版本的部署及使用,本篇介绍下最新版本HBase1.2.4的部署及使用,有部分区别,详见如下: 1. 环境准备: 1.需要在Hadoop[hadoop-2.7.3]  ...

随机推荐

  1. Linux SD卡驱动开发(四) —— SD 控制器之真正的硬件操作

    前面对SD卡控制器有了一个主要的介绍.事实上SD控制器层更过的意义是为core层提供一种操作SD卡硬件的一种方法.当然不同的控制器对硬件控制的方法不尽同样,可是他们终于都能像core层提交一个统一的封 ...

  2. C#常用数据类型间的转换

    数据类型有很多种,数据类型间的转换也是有很多的方法,如果不细心整理的话等到用的时候再查就会显得很浪费时间,所以决心整理出这篇博文.主要是讲解常用数据类型之间的转换方法以及常见数据类型所占字节数. 字节 ...

  3. RNA_seq GATK 最佳实践

    GATK处理DNA 水平的snp 经验比较成熟,而RNA 水平较少,所以可能会存在错误 目前的流程兼顾了假阳性(不是真的snp位点)和假阴性(该位点是snp,却没有检测到):后续会不断改善     G ...

  4. VueJs中 Class 与 Style 绑定

    绑定 HTML Class 尽管可以用 Mustache 标签绑定 class,比如 class="{{ className }}",但是我们不推荐这种写法和 v-bind:cla ...

  5. 一款纯HTML+CSS+JS富文本编辑器-handyeditor

    官网:http://he.catfish-cms.com/ 修改版本(修改一些BUG和图片上传服务器 点击下载: handyeditor富文本编辑器.zip): 图片上传接口上传类型: Content ...

  6. 深度学习 Deep Learning UFLDL 最新Tutorial 学习笔记 5：Softmax Regression

    Softmax Regression Tutorial地址:http://ufldl.stanford.edu/tutorial/supervised/SoftmaxRegression/ 从本节開始 ...

  7. LR进行接口测试

    其实无论用那种测试方法,接口测试的原理是通过测试程序模拟客户端向服务器发送请求报文,服务器接收请求报文后对相应的报文做出处理然后再把应答报文发送给客户端,客户端接收应答报文这一个过程. 方法一.用Lo ...

  8. 用Lua实现string的trim()方法

    function trim1(s) return (s:gsub("^%s*(.-)%s*$", "%1")) end -- from PiL2 20.4 fu ...

  9. mysql中显示当前数据库下的所有表，包括视图。

    环境说明: mysql版本:5.5.57-log 操作系统:Red Hat Enterprise Linux Server release 6.6 (Santiago) 需求:查看当前数据库下所有的表 ...

  10. 利用MsChart控件绘制多曲线图表（转载）

    在.Net4.0框架中,微软已经将Mschart控件集成了进来,以前一直在web下面用过,原来winform下的Mschart控件更加简单更加方便,今天我们用mschart绘制一个多曲线图,话不多说, ...