hadoop---wordcount命令
[zznu@master file]$ hadoop jar ~/hadoop-2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.2.jar wordcount /inputfile output
16/04/11 22:31:02 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.222.122:18040
16/04/11 22:31:03 INFO input.FileInputFormat: Total input paths to process : 2
16/04/11 22:31:03 INFO mapreduce.JobSubmitter: number of splits:2
16/04/11 22:31:03 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1460438506725_0002
16/04/11 22:31:03 INFO impl.YarnClientImpl: Submitted application application_1460438506725_0002
16/04/11 22:31:04 INFO mapreduce.Job: The url to track the job: http://master:18088/proxy/application_1460438506725_0002/
16/04/11 22:31:04 INFO mapreduce.Job: Running job: job_1460438506725_0002
16/04/11 22:31:13 INFO mapreduce.Job: Job job_1460438506725_0002 running in uber mode : false
16/04/11 22:31:13 INFO mapreduce.Job: map 0% reduce 0%
16/04/11 22:31:26 INFO mapreduce.Job: map 100% reduce 0%
16/04/11 22:31:34 INFO mapreduce.Job: map 100% reduce 100%
16/04/11 22:31:34 INFO mapreduce.Job: Job job_1460438506725_0002 completed successfully
16/04/11 22:31:34 INFO mapreduce.Job: Counters: 49
File System Counters
FILE: Number of bytes read=67
FILE: Number of bytes written=290851
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=237
HDFS: Number of bytes written=25
HDFS: Number of read operations=9
HDFS: Number of large read operations=0
HDFS: Number of write operations=2
Job Counters
Launched map tasks=2
Launched reduce tasks=1
Data-local map tasks=2
Total time spent by all maps in occupied slots (ms)=21525
Total time spent by all reduces in occupied slots (ms)=6185
Total time spent by all map tasks (ms)=21525
Total time spent by all reduce tasks (ms)=6185
Total vcore-seconds taken by all map tasks=21525
Total vcore-seconds taken by all reduce tasks=6185
Total megabyte-seconds taken by all map tasks=22041600
Total megabyte-seconds taken by all reduce tasks=6333440
Map-Reduce Framework
Map input records=2
Map output records=6
Map output bytes=61
Map output materialized bytes=73
Input split bytes=200
Combine input records=6
Combine output records=5
Reduce input groups=3
Reduce shuffle bytes=73
Reduce input records=5
Reduce output records=3
Spilled Records=10
Shuffled Maps =2
Failed Shuffles=0
Merged Map outputs=2
GC time elapsed (ms)=267
CPU time spent (ms)=2860
Physical memory (bytes) snapshot=515694592
Virtual memory (bytes) snapshot=2516971520
Total committed heap usage (bytes)=257171456
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=37
File Output Format Counters
Bytes Written=25
[zznu@master file]$ hadoop fs -ls output
Found 2 items
-rw-r--r-- 1 zznu supergroup 0 2016-04-11 22:31 output/_SUCCESS
-rw-r--r-- 1 zznu supergroup 25 2016-04-11 22:31 output/part-r-00000
[zznu@master file]$
hadoop---wordcount命令的更多相关文章
- Hadoop Shell命令大全
hadoop支持命令行操作HDFS文件系统,并且支持shell-like命令与HDFS文件系统交互,对于大多数程序猿/媛来说,shell-like命令行操作都是比较熟悉的,其实这也是Hadoop的极大 ...
- hadoop CLASSNAME命令使用注意点
Hadoop中可是使用hadoop CLASSNAME命令.这个CLASSNAME就是你写好的类名.hadoop CLASSNAME命令类似于java classname. 使用hadoop CLAS ...
- 【Hadoop篇】--Hadoop常用命令总结
一.前述 分享一篇hadoop的常用命令的总结,将常用的Hadoop命令总结如下. 二.具体 1.启动hadoop所有进程start-all.sh等价于start-dfs.sh + start-yar ...
- 在执行hadoop fs命令时,出现WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable错误
错误呈现: 解决过程: (参考链接:https://www.cnblogs.com/kevinq/p/5103653.html) 1.输出hadoop的详细日志,并执行hadoop fs命令来查看错误 ...
- hadoop 管理命令dfsadmin
hadoop 管理命令dfsadmin dfsadmin 命令用于管理HDFS集群,这些命令常用于管理员. 1. (Safemode)安全模式 动作 命令 把集群切换到安全模式 bin/hdfs df ...
- Hadoop常用命令总结
一.前述 分享一篇hadoop的常用命令的总结,将常用的Hadoop命令总结如下. 二.具体 1.启动hadoop所有进程start-all.sh等价于start-dfs.sh + start-yar ...
- hadoop 常见 命令
一 hadoop namenode 命令 1 格式化namanode 磁盘 hadoop namenode -format 二 hadoop fs 命令 和 linux 命令 非常类似 ...
- 大数据之路week06--day07(Hadoop常用命令)
一.前述 分享一篇hadoop的常用命令的总结,将常用的Hadoop命令总结如下. 二.具体 1.启动hadoop所有进程start-all.sh等价于start-dfs.sh + start-yar ...
- hadoop目录命令
下面是经常使用到的,以此记录备忘 1.查看hadoop目录 命令: hadoop fs -ls / 2.创建目录 命令:hadoop fs -mkdir /目录名 3.将文件上传hadoop中(也就是 ...
- Hadoop常用命令及基本概念
HADOOP 是什么? 分布式计算开源框架,其核心组件为:HDFS.MAPREDUCE.YARN Hadoop各个功能模块的理解 1. HDFS模块 HDFS负责大数据的存储,通过将大文件分块后进行分 ...
随机推荐
- 为什么switch...case语句比if...else执行效率高
在C语言中,教科书告诉我们switch...case...语句比if...else if...else执行效率要高,但这到底是为什么呢?本文尝试从汇编的角度予以分析并揭晓其中的奥秘. 第一步,写一个d ...
- java自带的监控工具VisualVM(二)远程监控
ps:尝试了网上的几个网友提供的方法,始终不得其法,汇总后,终于尝试成功!将一些需要注意的细节也记录下来以后备用! 我们经常需要对我们的开发的软件做各种测试, 软件对系统资源的使用情况更是不可少, 目 ...
- BZOJ 1303 【CQOI2009】中位数图
baidu了一下bzoj水题列表...找到这道题. 题目大意:给定一个数t,在给定的一段包含1-n的序列中找出多少个长度为奇数子序列的中位数为t. 第一眼没看数据范围,于是开心的打了一个O(n^3 ...
- angularjs-xeditable整合typeahead完成智能提示
按照需求,需要在angularjs的xeditable中加入typeahead,来完成智能提示,并且在选择后,把该条数据的其他信息也显示在此行中,于是做了一下的测试修改. 当然,既然用了xeditab ...
- CodeForces 669C Little Artem and Matrix GNU
模拟. 把操作记录一下,倒着复原回去. #pragma comment(linker, "/STACK:1024000000,1024000000") #include<cs ...
- HDU 4520 小Q系列故事——最佳裁判
Time Limit : 500/200ms (Java/Other) Memory Limit : 65535/32768K (Java/Other) Problem Description 过 ...
- [河南省ACM省赛-第三届] 聪明的kk (nyoj 171)
题目链接:http://acm.nyist.net/JudgeOnline/problem.php?pid=171 动态规划: d(i,j) = max{d(i-1, j), d(i, j-1)}+m ...
- DataGrid 导出数据到 Excel
Private Sub GridToExl_Click() On Error Resume Next If DataGrid1.Columns.Count = 0 Then MsgBox " ...
- maven发布本地包,eclipse-maven集成tomcat7热部署项目
国内maven 库 :http://maven.aliyun.com/nexus/content/groups/public/ maven安装不在中央仓库的jar包: mvn install:inst ...
- LINQ to Entities does not recognize the method 'System.DateTime ToDateTime(System.String)' method
System.Data.Objects.EntityFunctions和System.Data.Objects.SqlClient.SqlFunctions中的方法进行比较,如下 where Syst ...