mongodb mapredReduce 多个条件分组(group by)
from:https://my.oschina.net/chiyong/blog/289138
Mongodb 没有传统数据库的group函数,如果分组需要走MapReduce。这种MR与HadoopMR类似。下面看看Mongodb 的分组实现
现在又一张 表它的数据格式如下:
{
"_id" : ObjectId("53b224e0a1ae72328a57702c"),
"title" : "SECJ0118E",
"criteria" : "未找到对应的错误码",
"actual" : "1",
"effect" : "可能引起重大问题",
"suggestion" : "请专家提供意见",
"severity" : "Normal",
"status" : "NotOK",
"rtype" : "FormLoginExte",
"comment" : "[8/2/12 17:28:21:231 GMT+08:00] 0000001e FormLoginExte E SECJ0118E: Authentication error during authentication for user rpt",
"category" : "logs",
"time" : "0008-02-12 17:28:21"
}
{
"_id" : ObjectId("53b224e0a1ae72328a577052"),
"title" : "",
"criteria" : "未找到对应的错误码",
"actual" : "1",
"effect" : "可能引起重大问题",
"suggestion" : "请专家提供意见",
"severity" : "Normal",
"status" : "NotOK",
"rtype" : "servlet",
"comment" : "[8/2/12 19:04:41:891 CST] 0000000b servlet E com.ibm.ws.webcontainer.servlet.ServletWrapper init Uncaught.init.exception.thrown.by.servlet",
"category" : "logs",
"time" : "0008-02-12 19:04:41"
}
{
"_id" : ObjectId("53b224e0a1ae72328a576fdc"),
"title" : "系统资源设置[processes]",
"criteria" : "unlimited",
"actual" : "unlimited",
"effect" : "如果对用户资源做了limits限制,有可能造成应用运行失败或系统性能下降。",
"suggestion" : "建议修改/etc/security/limits,编辑root相关参数部分都为-1。",
"severity" : "None",
"status" : "OK",
"rtype" : "系统参数设置检查",
"comment" : "",
"category" : "params"
}
1:单个条件分组
(1) 现在我们根据title进行分组 并且统计每个组的数量
db.runCommand({ mapreduce: "check_result",
map : function Map() {
//emit 函数中的key是唯一的,是分组条件 这里把title传递过来。
emit(
this.title,
{count: 1}
);
},
reduce : function Reduce(key, values) {
total=0;//定义一个变量total , values是一个数组
for( var i in values){
total +=values[i].count
}
return {"count":total};
},
finalize : function Finalize(key, reduced) {
return reduced;
},
out : { inline : 1 }
});
结果如下(结果中的_id键就是要分组的title 。value是分组后的值):
{ "_id" : "" , "value" : { "count" : 113.0}}
{ "_id" : "/tmp是否设置了t标志位" , "value" : { "count" : 21.0}}
{ "_id" : "ASYN0080W" , "value" : { "count" : 120.0}}
{ "_id" : "AppServer的JVM堆最大值" , "value" : { "count" : 6.0}}
{ "_id" : "AppServer的JVM堆最小值" , "value" : { "count" : 6.0}}
{ "_id" : "AppServer的JVM标准输出日志切换周期" , "value" : { "count" : 6.0}}
{ "_id" : "AppServer的JVM标准输出日志回滚类型" , "value" : { "count" : 6.0}}
{ "_id" : "AppServer的JVM标准错误日志切换周期" , "value" : { "count" : 6.0}}
{ "_id" : "AppServer的JVM标准错误日志回滚类型" , "value" : { "count" : 6.0}}
{ "_id" : "AppServer的WebContainer线程池最大值" , "value" : { "count" : 6.0}}
{ "_id" : "AppServer的WebContainer线程池最小值" , "value" : { "count" : 6.0}}
{ "_id" : "AppServer的通用JVM参数" , "value" : { "count" : 6.0}}
{ "_id" : "AppServer的通用JVM参数-SystemGC" , "value" : { "count" : 6.0}}
{ "_id" : "Audit是否打开" , "value" : { "count" : 21.0}}
{ "_id" : "CWPKI0041W" , "value" : { "count" : 65.0}}
{ "_id" : "CWPMC0017W" , "value" : { "count" : 7.0}}
{ "_id" : "CWSAA0037W" , "value" : { "count" : 13.0}}
{ "_id" : "Could not invoke an operation on object" , "value" : { "count" : 21.0}}
{ "_id" : "DCSV0000W" , "value" : { "count" : 4.0}}
{ "_id" : "DCSV1115W" , "value" : { "count" : 137.0}}
2:多个条件分组
(1) 现在我们根据title,status,severity进行分组 并且统计每个组的数量
db.runCommand({ mapreduce: "check_result",
map : function Map() {
//emit 函数中的key是唯一的,是分组条件
emit(
{"title":this.title,"status":this.status,"serverity":this.severity}
,
{count: 1}
);
},
reduce : function Reduce(key, values) {
total=0;//定义一个变量total , values是一个数组
for( var i in values){
total +=values[i].count
}
return {"count":total};
},
finalize : function Finalize(key, reduced) {
return reduced;
},
out : { inline : 1 }
});
输出结果如下格式化:
{ "_id" : { "title" : "" , "status" : "NotOK"} , "value" : { "count" : 113.0}}
{ "_id" : { "title" : "/tmp是否设置了t标志位" , "status" : "NotOK"} , "value" : { "count" : 21.0}}
{ "_id" : { "title" : "ASYN0080W" , "status" : "NotOK"} , "value" : { "count" : 120.0}}
{ "_id" : { "title" : "AppServer的JVM堆最大值" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "AppServer的JVM堆最小值" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "AppServer的JVM标准输出日志切换周期" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "AppServer的JVM标准输出日志回滚类型" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "AppServer的JVM标准错误日志切换周期" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "AppServer的JVM标准错误日志回滚类型" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "AppServer的WebContainer线程池最大值" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "AppServer的WebContainer线程池最小值" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "AppServer的通用JVM参数" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "AppServer的通用JVM参数-SystemGC" , "status" : "NotOK"} , "value" : { "count" : 6.0}}
{ "_id" : { "title" : "Audit是否打开" , "status" : "NotOK"} , "value" : { "count" : 21.0}}
{ "_id" : { "title" : "CWPKI0041W" , "status" : "NotOK"} , "value" : { "count" : 65.0}}
{ "_id" : { "title" : "CWPMC0017W" , "status" : "NotOK"} , "value" : { "count" : 7.0}}
{ "_id" : { "title" : "CWSAA0037W" , "status" : "NotOK"} , "value" : { "count" : 13.0}}
{ "_id" : { "title" : "Could not invoke an operation on object" , "status" : "NotOK"} , "value" : { "count" : 21.0}}
{ "_id" : { "title" : "DCSV0000W" , "status" : "NotOK"} , "value" : { "count" : 4.0}}
{ "_id" : { "title" : "DCSV1115W" , "status" : "NotOK"} , "value" : { "count" : 137.0}}
mongodb mapredReduce 多个条件分组(group by)的更多相关文章
- Oracle 表分组 group by和模糊查询like
分组group by写法 select 字段名 from 表名 group by 字段名 查询这个字段名里的种类分组后可以加聚合函数select 字段名,聚合函数 from 表名 group by 字 ...
- mysql分组GROUP BY常用sql
数据分组 GROUP BY GROUP BY可以根据一个或多个字段进行分组. 比如,根据prod_id分组: SELECT prod_id ,user_id FROM products GROUP B ...
- 第17课-数据库开发及ado.net 聚合函数,模糊查询like,通配符.空值处理.order by排序.分组group by-having.类型转换-cast,Convert.union all; Select 列 into 新表;字符串函数;日期函数
第17课-数据库开发及ado.net 聚合函数,模糊查询like,通配符.空值处理.order by排序.分组group by-having.类型转换-cast,Convert.union all; ...
- spring data jpa条件分组查询及分页
原book对象 package com.shaying.domain; import javax.persistence.Column; import javax.persistence.Entity ...
- sql-多表查询JOIN与分组GROUP BY
一.内部连接:两个表的关系是平等的,可以从两个表中获取数据.用ON表示连接条件 SELECT A.a,B.b FROM At AS A INNER JOINT Bt AS B ON A.m=B.n ...
- js将数组根据条件分组
//将数组根据条件分组 function getTreeDateByParam(list, param, fun){ var data = {}; if(list && list.le ...
- MySQL数据分组Group By 和 Having
现有以下的学生信息表: 若果现在想计算每个班的平均年龄,使用where的操作如下: SELECT Cno AS 班级, AVG(Sage) AS 平均年龄 FROM stu ; 这样的话,有多少个班就 ...
- TFS二次开发10——分组(Group)和成员(Member)
TFS SDK 10 ——分组(Group)和成员(Member) 这篇来介绍怎样读取TFS服务器上的用户信息 首先TFS默认有如下分组(Group): SharePoint Web Applicat ...
- 【SQL】分组数据,过滤分组-group by , having
学习笔记,原文来自http://blog.csdn.net/robinjwong/article/details/24845125 创建分组 - GROUP BY 分组是在SELECT语句的GROUP ...
随机推荐
- canvas 线性规划
小结: 1.线性规划 <!DOCTYPE html> <html> <head> <meta charset="utf-8"> &l ...
- extundelete fuser
[extundelete ] fuser -mv /dev/sda6 https://blog.csdn.net/xwl145/article/details/38896699 多列排序[sort]h ...
- Chap2:区块链基本技术[《区块链中文词典》维京&甲子]
- iOS只给矩形两个边加圆角
- (void)updataTopCornerRadius { CGRect clipRect = CGRectMake(, , self.headPhotoIv.width, self.headPh ...
- 20165225 2017-2018-2《Java程序设计》课程总结
20165225 2017-2018-2<Java程序设计>课程总结 - 每周作业链接汇总: 预备作业一:我期待的师生关系 预备作业二:学习基础和C语言基础调查 预备作业三:linux安装 ...
- Gson使用技巧
1. CharMatcher String serviceUrl = CharMatcher.is('/').trimTrailingFrom(ConfigHelper.metaServiceUrl( ...
- Web Deploy远程部署配置图解
原文链接:https://jingyan.baidu.com/album/642c9d34e614de644a46f783.html
- 根据xml配置使用反射动态生成对象
web.xml <?xml version="1.0" encoding="UTF-8"?> <web-app xmlns="htt ...
- node2vec应用记录
1.已有写好的python代码,可以直接下载调用,GitHub链接https://github.com/aditya-grover/node2vec/blob/master/requirements. ...
- dedecms站内搜索页面调用最新文章
在页面中调用最新文章列表可以使新发布的文章更快被收录,如何在dedecms站内搜索页面调用最新文章呢? 1.登陆系统后台,进入“模板——模板管理——自定义宏标记”,点击“智能标记向导”进入智能标记生成 ...