Elasticsearch系列(二)--query、filter、aggregations
本文基于ES6.4版本，我也是出于学习阶段，对学习内容做个记录，如果文中有错误，请指出。
实验数据：
index：book
type：novel
mappings：
{
"mappings": {
"novel": {
"dynamic": "false",
"properties": {
"word_count": {
"type": "integer"
},
"author": {
"type": "keyword"
},
"title": {
"type": "text"
},
"publish_date": {
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis",
"type": "date"
}
}
}
}
}
通过put创建索引，使用head可视化界面，数据如下：
Elasticsearch的查询分为：
1、子条件查询：查询特定字段的特定值
Query context
查询过程中，除了判断Document是否满足条件，还会计算出_score表示匹配程度，数值越大，证明匹配程度越高
1、查询全部：/book/novel/_search
"hits": {
"total": 10,
"max_score": 1.0,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "5",
"_score": 1.0,
"_source": {
"title": "永夜君王",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "烟雨江南"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "8",
"_score": 1.0,
"_source": {
"title": "万古令",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "听奕"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "9",
"_score": 1.0,
"_source": {
"title": "天帝传",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "飞天鱼"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "10",
"_score": 1.0,
"_source": {
"title": "剑来",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "烽火戏诸侯"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "2",
"_score": 1.0,
"_source": {
"title": "完美世界",
"word_count": "130000",
"publish_date": "2017-03-01",
"author": "辰东"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "4",
"_score": 1.0,
"_source": {
"title": "民国谍影",
"word_count": "110000",
"publish_date": "2019-03-01",
"author": "寻青藤"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "6",
"_score": 1.0,
"_source": {
"title": "遮天",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "辰东"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_score": 1.0,
"_source": {
"title": "万古神帝",
"word_count": "30000",
"publish_date": "2017-01-01",
"author": "飞天鱼"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "7",
"_score": 1.0,
"_source": {
"title": "圣墟",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "辰东"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "3",
"_score": 1.0,
"_source": {
"title": "星辰变",
"word_count": "100000",
"publish_date": "2018-03-01",
"author": "我吃西红柿"
}
}
]
}
2、查询id为1的数据：/book/novel/1
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_version": 1,
"found": true,
"_source": {
"title": "万古神帝",
"word_count": "30000",
"publish_date": "2017-01-01",
"author": "飞天鱼"
}
}
3、只查询title和author字段：/1?_source=title,author
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_version": 1,
"found": true,
"_source": {
"author": "飞天鱼",
"title": "万古神帝"
}
}
4、只是显示_source部分：/book/novel/1/_source
{
"title": "万古神帝",
"word_count": "30000",
"publish_date": "2017-01-01",
"author": "飞天鱼"
}
5、筛选单字段查询：/book/novel/_search
{
"query": {
"match": {
"author": "飞天鱼"
}
}
}
"hits": {
"total": 2,
"max_score": 1.2039728,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "9",
"_score": 1.2039728,
"_source": {
"title": "天帝传",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "飞天鱼"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_score": 0.6931472,
"_source": {
"title": "万古神帝",
"word_count": "30000",
"publish_date": "2017-01-01",
"author": "飞天鱼"
}
}
]
}
6、limit：我们查询到2条数据，如果我们只想得到第一条数据，可以使用from和size联合查询
{
"query": {
"match": {
"author": "飞天鱼"
}
},
"from": 0,
"size": 1
}
"hits": {
"total": 2,
"max_score": 1.2039728,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "9",
"_score": 1.2039728,
"_source": {
"title": "天帝传",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "飞天鱼"
}
}
]
}
{
"query": {
"match": {
"author": "辰东"
}
},
"sort": [
{
"word_count": {
"order": "desc"
}
}
]
}
"hits": {
"total": 3,
"max_score": null,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "2",
"_score": null,
"_source": {
"title": "完美世界",
"word_count": "130000",
"publish_date": "2017-03-01",
"author": "辰东"
},
"sort": [
130000
]
},
{
"_index": "book",
"_type": "novel",
"_id": "6",
"_score": null,
"_source": {
"title": "遮天",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "辰东"
},
"sort": [
110000
]
},
{
"_index": "book",
"_type": "novel",
"_id": "7",
"_score": null,
"_source": {
"title": "圣墟",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "辰东"
},
"sort": [
110000
]
}
]
}
8、其余匹配match_phrase：
query、match的方式本质上就是模糊查询，而且中文会自动分词到最大粒度，可以看到会查询到只要匹配任意一个字都是可以的
{
"query": {
"match": {
"title": "万古神帝"
}
}
}
"hits": {
"total": 3,
"max_score": 2.439878,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_score": 2.439878,
"_source": {
"title": "万古神帝",
"word_count": "30000",
"publish_date": "2017-01-01",
"author": "飞天鱼"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "8",
"_score": 2.4079456,
"_source": {
"title": "万古令",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "听奕"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "9",
"_score": 1.2039728,
"_source": {
"title": "天帝传",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "飞天鱼"
}
}
]
}
所以这里有了其余匹配match_phrase，结果只有完全包含"万古神帝"的title才可以被查询到
{
"query": {
"match_phrase": {
"title": "万古神帝"
}
}
}
"hits": {
"total": 1,
"max_score": 2.439878,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_score": 2.439878,
"_source": {
"title": "万古神帝",
"word_count": "30000",
"publish_date": "2017-01-01",
"author": "飞天鱼"
}
}
]
}
9、多条件查询multi_match：查询title或者author包含"万古神帝"的数据
{
"query": {
"multi_match": {
"query": "万古神天",
"fields": ["title","author"]
}
}
}
"hits": {
"total": 4,
"max_score": 2.4079456,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "8",
"_score": 2.4079456,
"_source": {
"title": "万古令",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "听奕"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "1",
"_score": 1.8299085,
"_source": {
"title": "万古神帝",
"word_count": "30000",
"publish_date": "2017-01-01",
"author": "飞天鱼"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "9",
"_score": 1.2039728,
"_source": {
"title": "天帝传",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "飞天鱼"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "6",
"_score": 1.1727304,
"_source": {
"title": "遮天",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "辰东"
}
}
]
}
10、语法查询query_string：
{
"query": {
"query_string": {
"query": "万古"
}
}
}
这里和match没有区别，query可以使用AND和OR，match的filed也可以，注意这里一定是大写，小写就被当做搜索的内容了
{
"query": {
"query_string": {
"query": "万古 OR 剑来"
}
}
}
{
"query": {
"match": {
"title": "万古 OR 剑来"
}
}
}
指定fields：
{
"query": {
"query_string": {
"query": "万古 OR 剑来 OR 辰东 ",
"fields": ["author","title"]
}
}
}
11、精确匹配term：
title为text类型，author为keyword类型，实验发现查询title只有是单个字的时候才能匹配(精确匹配查不到数据)，而author必须是精确匹配
例如：title不支持精确匹配，支持模糊查询(而且是单个字才可以，多个字照样查不到数据)
{
"query": {
"term": {
"title": "剑来"
}
}
}
如果只是查询一个字就可以
{
"query": {
"term": {
"title": "来"
}
}
}
"hits": {
"total": 1,
"max_score": 1.3940737,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "10",
"_score": 1.3940737,
"_source": {
"title": "剑来",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "烽火戏诸侯"
}
}
]
}
查询author字段：有三条数据
{
"query": {
"term": {
"author": "辰东"
}
}
}
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "7",
"_score": 0.6931472,
"_source": {
"title": "圣墟",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "辰东"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "2",
"_score": 0.47000363,
"_source": {
"title": "完美世界",
"word_count": "130000",
"publish_date": "2017-03-01",
"author": "辰东"
}
},
{
"_index": "book",
"_type": "novel",
"_id": "6",
"_score": 0.47000363,
"_source": {
"title": "遮天",
"word_count": "110000",
"publish_date": "2015-03-01",
"author": "辰东"
}
}
]
}
author不知道模糊查询：下面结果为null
{
"query": {
"term": {
"author": "东"
}
}
}
12、范围查找range：包括integer和日期类型，日期支持now函数，也就是当前日期
{
"query": {
"range": {
"word_count": {
"gt": 110000,
"lte": 130000
}
}
}
}
"hits": {
"total": 1,
"max_score": 1.0,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "2",
"_score": 1.0,
"_source": {
"title": "完美世界",
"word_count": "130000",
"publish_date": "2017-03-01",
"author": "辰东"
}
}
]
}
Filter context
查询过程中，只是判断Document是否满足条件，只有yes or no。用来做数据过滤，而且ES还会对结果进行缓存，效率相对query更高一点
{
"query": {
"bool": {
"filter": {
"term": {
"word_count": 130000
}
}
}
}
}
"hits": {
"total": 1,
"max_score": 0.0,
"hits": [
{
"_index": "book",
"_type": "novel",
"_id": "2",
"_score": 0.0,
"_source": {
"title": "完美世界",
"word_count": "130000",
"publish_date": "2017-03-01",
"author": "辰东"
}
}
]
}
2、复合条件查询：组合子条件查询
1、固定分数查询：不支持match，支持filter
{
"query": {
"constant_score": {
"filter": {
"match": {
"title": "天帝传"
}
}
}
}
}
{
"query": {
"constant_score": {
"filter": {
"match": {
"title": "天帝传"
}
},
"boost": 2
}
}
}
2、bool查询：
should：就是or的关系
{
"query": {
"bool": {
"should": [
{
"match": {
"author": "辰东"
}
},
{
"match": {
"title": "天帝传"
}
}
]
}
}
}
must：相当于and
{
"query": {
"bool": {
"must": [
{
"match": {
"author": "辰东"
}
},
{
"match": {
"title": "天帝传"
}
}
]
}
}
}
must_not：相当于<>
{
"query": {
"bool": {
"must_not": {
"term": {
"author": "辰东"
}
}
}
}
}
bool查询也可以使用filter：
{
"query": {
"bool": {
"must": [
{
"match": {
"author": "辰东"
}
},
{
"match": {
"title": "天帝传"
}
}
],
"filter": [
{
"term": {
"word_count": 110000
}
}
]
}
}
}
aggregations：
{
"aggs": {
"group_by_author": {
"terms": {
"field": "author"
}
}
}
}
"aggregations": {
"group_by_author": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "辰东",
"doc_count": 3
},
{
"key": "飞天鱼",
"doc_count": 2
},
{
"key": "听奕",
"doc_count": 1
},
{
"key": "寻青藤",
"doc_count": 1
},
{
"key": "我吃西红柿",
"doc_count": 1
},
{
"key": "烟雨江南",
"doc_count": 1
},
{
"key": "烽火戏诸侯",
"doc_count": 1
}
]
}
}
支持多聚合结果：
{
"aggs": {
"group_by_author": {
"terms": {
"field": "author"
}
},
"group_by_word_count": {
"terms": {
"field": "word_count"
}
}
}
}
aggregations除了支持term，还有stats、min、max、avg等
{
"aggs": {
"group_by_author": {
"stats": {
"field": "word_count"
}
}
}
}
"aggregations": {
"group_by_author": {
"count": 10,
"min": 30000.0,
"max": 130000.0,
"avg": 103000.0,
"sum": 1030000.0
}
}
avg：
{
"aggs": {
"group_by_author": {
"avg": {
"field": "word_count"
}
}
}
}
Elasticsearch系列(二)--query、filter、aggregations的更多相关文章
- elasticsearch系列二：索引详解（快速入门、索引管理、映射详解、索引别名）
一.快速入门 1. 查看集群的健康状况 http://localhost:9200/_cat http://localhost:9200/_cat/health?v 说明:v是用来要求在结果中返回表头 ...
- Elasticsearch学习笔记（十二）filter与query
一.keyword 字段和keyword数据类型 1.测试准备数据 POST /forum/article/_bulk { "index": { "_id" ...
- WEB API 系列（二） Filter的使用以及执行顺序
在WEB Api中,引入了面向切面编程(AOP)的思想,在某些特定的位置可以插入特定的Filter进行过程拦截处理.引入了这一机制可以更好地践行DRY(Don’t Repeat Yourself)思想 ...
- Web API系列（二） Filter的使用以及执行顺序
在WEB Api中,引入了面向切面编程(AOP)的思想,在某些特定的位置可以插入特定的Filter进行过程拦截处理.引入了这一机制可以更好地践行DRY(Don’t Repeat Yourself)思想 ...
- Elasticsearch系列---常见搜索方式与聚合分析
概要 本篇主要介绍常见的6种搜索方式.聚合分析语法,基本是上机实战,可以和关系型数据库作对比,如果之前了解关系型数据库,那本篇只需要了解搜索和聚合的语法规则就可以了. 搜索响应报文 以上篇建立的mus ...
- Elasticsearch入门教程(六)：Elasticsearch查询(二)
原文:Elasticsearch入门教程(六):Elasticsearch查询(二) 版权声明:本文为博主原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接和本声明. 本文链接:h ...
- Wireshark入门与进阶系列(二)
摘自http://blog.csdn.net/howeverpf/article/details/40743705 Wireshark入门与进阶系列(二) “君子生非异也,善假于物也”---荀子 本文 ...
- Android高效率编码-第三方SDK详解系列(二)——Bmob后端云开发，实现登录注册，更改资料，修改密码，邮箱验证，上传，下载，推送消息，缩略图加载等功能
Android高效率编码-第三方SDK详解系列(二)--Bmob后端云开发,实现登录注册,更改资料,修改密码,邮箱验证,上传,下载,推送消息,缩略图加载等功能 我的本意是第二篇写Mob的shareSD ...
- 搜索引擎ElasticSearchV5.4.2系列二之ElasticSearchV5.4.2+kibanaV5.4.2+x-packV5.4.2安装
相关博文: 搜索引擎ElasticSearchV5.4.2系列一之ES介绍 搜索引擎ElasticSearchV5.4.2系列二之ElasticSearchV5.4.2+klanaV5.4.2+x-p ...
随机推荐
- spring boot项目开发中遇到问题，持续更新
1.JPA中EntityManager不能执行建表语句,提示要加事务Error:javax.persistence.TransactionRequiredException: Executing an ...
- BZOJ 2660 (BJOI 2012) 最多的方案
Description 第二关和很出名的斐波那契数列有关,地球上的OIer都知道:F1=1, F2=2, Fi = Fi-1 + Fi-2,每一项都可以称为斐波那契数.现在给一个正整数N,它可以写成一 ...
- （转）C# 使用UDP组播实现局域网桌面共享
转:http://www.cnblogs.com/mobwiz/p/3715743.html 最近需要在产品中加入桌面共享的功能,暂时不用实现远程控制:参考了园子里的一些文章,加入了一些自己的修改. ...
- 海量数据解决思路之Hash算法
海量数据解决思路之Hash算法 一.概述 本文将粗略讲述一下Hash算法的概念特性,里边会结合 分布式系统负载均衡 实例对Hash的一致性做深入探讨.另外,探讨一下Hash算法在海量数据处理方案中 ...
- Laravel5.4中自定义404等错误页面
1.在resources/views/下简历文件夹error,在error文件中建立"404.blade.php文件". <!DOCTYPE html PUBLIC &quo ...
- JAVA判断一个对象生存还是死亡
JAVA中判断一个对象是否死亡的算法有两种: 引用计数算法 可达性分析算法 一.引用计数算法所谓引用计数算法就是,给一个对象定义一个引用计数器,每当该对象被引用一次引用计数器就加1,如果一个对象的引用 ...
- Celery - 异步任务 , 定时任务 , 周期任务
1.什么是Celery?Celery 是芹菜Celery 是基于Python实现的模块, 用于执行异步定时周期任务的其结构的组成是由 1.用户任务 app 2.管道 broker 用于存储 ...
- Facebook分布式框架—Thrift介绍。
Thrift介绍 Thrift是一个分布式RPC框架,用来进行可扩展且跨语言的服务的开发.它结合了功能强大的软件堆栈和代码生成引擎,以构建在 C++, Java, Python, PHP, Ruby, ...
- SpringAOP中的aop:config标签
我们使用Spring的AOP功能的时候发现,我们使用普通的配置方式的时候,我们无法精确的确定将切面类中的哪个方法切入到哪个切入点上, 所以我们可以使用aop的专用标签来完成相关的配置.其中主要表现是使 ...
- POJ 3449 /// 判断线段相交
题目大意: 给出多个多边形及其编号 按编号顺序输出每个多边形与其相交的其他多边形编号 注意一个两个多个的不同输出 将每个多边形处理成多条边 然后去判断与其他多边形的边是否相交 计算正方形另外两点的方法 ...