java整合Elasticsearch,实现crud以及高级查询的分页,范围,排序功能,泰文分词器的使用,分组,最大,最小,平均值,以及自动补全功能
//为index创建mapping,index相当于mysql的数据库,数据库里的表也要给各个字段创建类型,所以index也要给字段事先设置好类型:
使用postMan或者其他工具创建:(此处我使用postMan,创建一个名为shop的index,type是order-- type相等于mysql的表)
//这里的背景是一个订单表对应多个订单项表(商品信息),然后就将所有的订单和购买的商品信息存到ES,我这里的ES版本是6.4.2
//以下介绍的mapping字段分词器都是英文的,如果要使用中文分词器,就自行百度es如何配置中文分词器
put localthost:9200/shop
{
"mappings": {
"order": {
"dynamic": false,//设置为false的意思就是以后不能动态添加字段
"properties": {
"orderId": {
"type": "keyword", // 字符串类型只有text和keyword,text是会分词
"index": true //如果要以该字段作为搜索条件,就必须写成true
},
"orderAmount": {
"type": "double"
},
"orderTitle": {
"type": "text",
"analyzer": "english" //英文分词器,如果使用中文ik分词器,就自行百度怎么配置
},
"customerId": {
"type": "keyword",
"index": true
},
"orderSubmit": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
},
"payTime": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
},
"finshiTime": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
},
"expressAmount": {
"type": "double"
},
"deliveryTime": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
},
"couponAmount": {
"type": "double"
},
"score": {
"type": "integer"
},
"scoreAmount": {
"type": "double"
},
"orderStatus": {
"type": "integer"
},
"paymentStatus": {
"type": "integer"
},
"markertId": {
"type": "keyword",
"index": true
},
"shopId": {
"type": "keyword",
"index": true
},
"shopName": {
"type": "text",
"analyzer": "english"
},
"address": {
"type": "text",
"analyzer": "standard"
},
"recieveName": {
"type": "text",
"analyzer": "standard"
},
"phone": {
"type": "keyword",
"index": true
},
"itemList": {
"properties": {
"orderItemId": {
"type": "keyword",
"index": true
},
"itemId": {
"type": "keyword",
"index": true
},
"itemTitle": {
"type": "text",
"analyzer": "english"
},
"itemType": {
"type": "integer"
},
"quanity": {
"type": "integer"
},
"price": {
"type": "double"
},
"totalAmount": {
"type": "double"
}
}
}
}
}
}
}
//创建完成后,使用es可视化工具(我使用的是head)在浏览器登录可以看到对应的shop索引了
//ES的order对象结构如下:
public class EsOrderMast {
private String orderId;
private BigDecimal orderAmount;
private String orderTitle;
private String customerId;
private String orderSubmit;
private String payTime;
private String finshiTime;
private BigDecimal expressAmount;
private String deliveryTime;
private BigDecimal couponAmount;
private Integer score;
private BigDecimal scoreAmount;
private Integer orderStatus;
private Integer paymentStatus;
private String markertId;
private String shopId;
private String shopName;
private String address;
private String recieveName;
private String phone;
private List<EsOrderItem> itemList=new ArrayList<>();//订单的商品信息
}//省略了setter和getter
//订单的商品ES对象结构:
public class EsOrderItem { private String orderItemId; private String itemId; private String itemTitle; private Integer itemType; private Integer quanity; private BigDecimal price; private BigDecimal totalAmount;
}//省略setter和getter
//第一步创建java客户端,下面的例子都是通过该Util获取客户端client对象,当然也可以通过配置方式然后注入
public abstract class ESUtil { private static TransportClient client = null; public static TransportClient getClient() {
if(client==null){
Settings settings = Settings.builder().put("cluster.name", "yangxiaohui").build();//集群名称在es的配置文件可以找到
try {
client = new PreBuiltTransportClient(settings).addTransportAddress(new TransportAddress(InetAddress.getByName("localhost"), 9300));
} catch (UnknownHostException e) {
e.printStackTrace();
}
}
return client;
} }
//springboot整合es,使用时就可以通过Autowired注入的方式获取client对象,之后的例子获取客户端还是以EsUtil类获取,这里为了技术完整性附加上:
package com.example.demo.domain; import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.TransportAddress;
import org.elasticsearch.transport.client.PreBuiltTransportClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration; import javax.annotation.PostConstruct;
import java.net.InetAddress; @Configuration
public class ElasticsearchConfig implements InitializingBean { private static final Logger LOGGER = LoggerFactory.getLogger(ElasticsearchConfig.class); @Value("${tc.elasticsearch.cluster.name}")
private String clusterName; @Value("${tc.elasticsearch.port}")
private Integer port; @Value("${tc.elasticsearch.ip}")
private String host; /**
* Springboot整合Elasticsearch 在项目启动前设置一下的属性,防止报错
* 解决netty冲突后初始化client时还会抛出异常
* java.lang.IllegalStateException: availableProcessors is already set to [4], rejecting [4]
*/
@PostConstruct
void init() {
System.setProperty("es.set.netty.runtime.available.processors", "false");
} @Bean
public TransportClient getTransportClient() {
TransportClient client=null;
LOGGER.info("elasticsearch init.");
try {
Settings settings = Settings.builder()
.put("cluster.name", clusterName) //集群名字
.put("client.transport.sniff", true)//增加嗅探机制,找到ES集群
.put("thread_pool.search.size", 5).build();//增加线程池个数
client = new PreBuiltTransportClient(settings);
TransportAddress transportAddress = new TransportAddress(InetAddress.getByName(host), port);
client.addTransportAddresses(transportAddress);
LOGGER.info("elasticsearch init success.");
return client;
} catch (Exception e) {
throw new RuntimeException("elasticsearch init fail."+ e);
}
} public String getClusterName() {
return clusterName;
} public void setClusterName(String clusterName) {
this.clusterName = clusterName;
} public Integer getPort() {
return port;
} public void setPort(Integer port) {
this.port = port;
} public String getHost() {
return host;
} public void setHost(String host) {
this.host = host;
} @Override
public void afterPropertiesSet() throws Exception { }
}
//将数据库的信息查询出来,然后批量初始化到ES中
//批量新增
@Async //异步方式
public void initESOrderMast() {
List<EsOrderMast> esItemList = orderMastMapper.findEsItemList();
TransportClient client = ESUtil.getClient();
BulkRequest bulkRequest = new BulkRequest();
if(null!=esItemList && esItemList.size()>0){
for (EsOrderMast esOrderMast : esItemList) {
IndexRequest indexRequest = new IndexRequest("shop", "order", esOrderMast.getOrderId());
indexRequest.source(JSON.toJSONString(esOrderMast), XContentType.JSON);
bulkRequest.add(indexRequest);
}
}
ActionFuture<BulkResponse> bulk = client.bulk(bulkRequest);
System.out.println(bulk.isDone());
BulkResponse bulkItemResponses = bulk.actionGet();
System.out.println(bulkItemResponses.status().getStatus()); }
//下面是增删改查代码
//删除
public boolean deleteByOrderId(String orderId){
boolean flag =false;
TransportClient client = ESUtil.getClient();
ActionFuture<DeleteResponse> actionFuture = client.delete(new DeleteRequest("shop", "order", orderId));
flag = actionFuture.isDone();
DeleteResponse deleteResponse = actionFuture.actionGet();
System.out.println(deleteResponse.getResult().toString());
System.out.println(deleteResponse.status().getStatus());
return flag;
}
//查找
public EsOrderMast getOrderByOrderId(String orderId){
EsOrderMast esOrderMast=null;
TransportClient client = ESUtil.getClient();
GetRequestBuilder getRequestBuilder = client.prepareGet("shop", "order", orderId);
GetResponse response = getRequestBuilder.get();
String sourceAsString = response.getSourceAsString();
esOrderMast = JSON.parseObject(sourceAsString, EsOrderMast.class);
return esOrderMast;
}
//修改
public boolean update(String orderId,Integer orderStatus) {
EsOrderMast orderMast = getOrderByOrderId(orderId);
orderMast.setOrderStatus(orderStatus);
orderMast.setFinshiTime(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date()));
TransportClient client = ESUtil.getClient();
//UpdateResponse updateResponse = client.prepareUpdate("shopdb", "order", orderId).setDoc(JSON.toJSONString(orderMast), XContentType.JSON).execute().actionGet();
/* UpdateRequest request = new UpdateRequest("", "order", orderId);
request.doc(JSON.toJSON(orderMast),XContentType.JSON);*/ //client.update(request).get();
UpdateResponse updateResponse = client.prepareUpdate("shop", "order", orderId)
.setDoc(JSON.toJSON(orderMast),XContentType.JSON).get();
//System.out.println(update.actionGet().status().getStatus()); return updateResponse.status().getStatus()==200?true:false; }
//高级查询:分页,排序,范围查找:
先看高级查询请求对象:
public class EsQueryObject {
private String orderId; private String customerId; private String txtOrderTitle; private Integer orderStatus; private Integer paymentStatus; private String phone; private String recieveName; private String addresss; private String orderSubmitTime_S;
private String orderSubmitTime_E; private String payTime_S;
private String payTime_E; private BigDecimal minPayAmount; private BigDecimal maxPayAmount; private String shopId; private String itemId; private String itemTile; private Page page;
}
//高级查询方法:(es的分页默认从0开始,如果没有设置分页,或者没有设置每页大小,默认返回10条数据)
public List<EsOrderMast> findOrderList(EsQueryObject queryObject){
TransportClient client = ESUtil.getClient();
SearchRequestBuilder sourceBuilder=client.prepareSearch("shop");
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
if(!StringUtils.isEmpty(queryObject.getOrderId())){
boolQueryBuilder.must(QueryBuilders.termQuery("orderId",queryObject.getOrderId()));
}
if(!StringUtils.isEmpty(queryObject.getCustomerId())){
boolQueryBuilder.must(QueryBuilders.termQuery("customerId",queryObject.getCustomerId()));
}
if(!StringUtils.isEmpty(queryObject.getTxtOrderTitle())){
boolQueryBuilder.must(QueryBuilders.matchQuery("orderTitle",queryObject.getTxtOrderTitle()));
}
if(null!=queryObject.getOrderStatus() && queryObject.getOrderStatus()>=0){
boolQueryBuilder.must(QueryBuilders.termQuery("orderStatus",queryObject.getOrderStatus()));
}
if(null!=queryObject.getPaymentStatus() && queryObject.getPaymentStatus()>=0){
boolQueryBuilder.must(QueryBuilders.termQuery("paymentStatus",queryObject.getPaymentStatus()));
}
if(!StringUtils.isEmpty(queryObject.getPhone())){
boolQueryBuilder.must(QueryBuilders.termQuery("phone",queryObject.getPhone()));
}
if(!StringUtils.isEmpty(queryObject.getRecieveName())){
boolQueryBuilder.must(QueryBuilders.matchQuery("recieveName",queryObject.getRecieveName()));
}
if(!StringUtils.isEmpty(queryObject.getAddresss())){
boolQueryBuilder.must(QueryBuilders.matchQuery("addresss",queryObject.getAddresss()));
}
if(!StringUtils.isEmpty(queryObject.getShopId())){
boolQueryBuilder.must(QueryBuilders.termQuery("shopId",queryObject.getShopId()));
}
if(!StringUtils.isEmpty(queryObject.getItemId())){
boolQueryBuilder.must(QueryBuilders.termQuery("itemList.itemId",queryObject.getItemId()));
}
if(!StringUtils.isEmpty(queryObject.getItemTile())){
boolQueryBuilder.must(QueryBuilders.matchQuery("itemList.itemTitle",queryObject.getItemTile()));
}
if(!StringUtils.isEmpty(queryObject.getOrderSubmitTime_S())){
boolQueryBuilder.must(QueryBuilders.rangeQuery("orderSubmit").gte(queryObject.getOrderSubmitTime_S()));
}
if(!StringUtils.isEmpty(queryObject.getOrderSubmitTime_E())){
boolQueryBuilder.must(QueryBuilders.rangeQuery("orderSubmit").lte(queryObject.getOrderSubmitTime_E()));
}
if(!StringUtils.isEmpty(queryObject.getPayTime_S())){
boolQueryBuilder.must(QueryBuilders.rangeQuery("payTime").gte(queryObject.getPayTime_S()));
}
if(!StringUtils.isEmpty(queryObject.getPayTime_E())){
boolQueryBuilder.must(QueryBuilders.rangeQuery("payTime").lte(queryObject.getPayTime_E()));
}
if(null!=queryObject.getMinPayAmount()){
boolQueryBuilder.must(QueryBuilders.rangeQuery("orderAmount").gte(queryObject.getMinPayAmount().doubleValue()));
}
if(null!=queryObject.getMaxPayAmount()){
boolQueryBuilder.must(QueryBuilders.rangeQuery("orderAmount").lte(queryObject.getMaxPayAmount().doubleValue()));
}
sourceBuilder.setQuery(boolQueryBuilder);
sourceBuilder.addSort(SortBuilders.fieldSort("orderSubmit").order(SortOrder.ASC));//排序
/*
//有时想每个分组取一条数据,这样可以使用下面的方法
CollapseBuilder groupShopId = new CollapseBuilder("shopId"); //表示按shopId进行分组,然后去重,每组取一条
sourceBuilder.setCollapse(groupShopId);
CardinalityAggregationBuilder cab=AggregationBuilders.cardinality("group_count").field("shopId");//获取分组的组数,因为通过下面
hits.totalHits获取的总数不是分组的总数,这里可以统计分组的组数,在分页时可以有用
sourceBuilder.addAggregation(cab);*/
if(null!=queryObject.getPage()){
Page page = queryObject.getPage();
int pageSize = page.getPageSize()==0?10:page.getPageSize();
//分页中的setFrom是偏移量不是当前页数
sourceBuilder.setFrom((page.getPageNum() == 0 ? 0 : page.getPageNum() - 1) * page.getPageSize()).setSize(pageSize);
}
List<EsOrderMast> list = new ArrayList<>();
ActionFuture<SearchResponse> execute = sourceBuilder.execute();
SearchResponse searchResponse = null;
try {
searchResponse = execute.get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
// Aggregation group_count = searchResponse.getAggregations().get("group_count"); //打印分组的组数
//System.out.println(group_count.toString());
SearchHits hits = searchResponse.getHits();
System.out.println(searchResponse.getHits().totalHits);
if(null!=hits && hits.totalHits>0) {
for (SearchHit hit : hits) {
String sourceAsString = hit.getSourceAsString();
EsOrderMast orderMast = JSON.parseObject(sourceAsString, EsOrderMast.class);
list.add(orderMast);
}
}
return list;
}
//附上maven
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>6.4.2</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>6.4.2</version>
<exclusions>
<exclusion>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
</exclusion>
</exclusions>
</dependency>
//自定义分词器,如这里是泰文分词器:
{
"settings": {
"analysis": {
"analyzer": {
"thai_analyzer": { //自定义分词器名称
"type": "custom",
"tokenizer": "thai", //es自带thai文分词器,不用下载
"filter": [
"lowercase",
"asciifolding"
]
},
"caseSensitive": { //设置大小写是否敏感,对于部分词的字段大小写需要敏感
"filter": "lowercase",
"type": "custom",
"tokenizer": "keyword"
}
}
}
},
"mappings": {
"order": {
"dynamic": false,
"properties": {
"orderId": {
"type": "keyword",
"index": true
},
"orderAmount": {
"type": "double"
},
"orderTitle": {
"type": "text",
"analyzer": "thai_analyzer", //创建文档时使用泰文自定义分词器
"search_analyzer": "thai_analyzer" //查询时使用泰文分词器
},
"customerId": {
"type": "keyword",
"index": true
},
"orderSubmit": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
},
"payTime": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
},
"finshiTime": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
},
"expressAmount": {
"type": "double"
},
"deliveryTime": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
},
"couponAmount": {
"type": "double"
},
"score": {
"type": "integer"
},
"scoreAmount": {
"type": "double"
},
"orderStatus": {
"type": "integer"
},
"paymentStatus": {
"type": "integer"
},
"markertId": {
"type": "keyword",
"index": true
},
"shopId": {
"type": "keyword",
"index": true
},
"shopName": {
"type": "text",
"analyzer": "thai_analyzer",
"search_analyzer": "thai_analyzer"
},
"address": {
"type": "text",
"analyzer": "thai_analyzer",
"search_analyzer": "thai_analyzer"
},
"recieveName": {
"type": "text",
"analyzer": "thai_analyzer",
"search_analyzer": "thai_analyzer"
},
"phone": {
"type": "keyword",
"index": true
},
"itemList": {
"properties": {
"orderItemId": {
"type": "keyword",
"index": true
},
"itemId": {
"type": "keyword",
"index": true
},
"itemTitle": {
"type": "text",
"analyzer": "thai_analyzer",
"search_analyzer": "thai_analyzer"
},
"itemType": {
"type": "integer"
},
"quanity": {
"type": "integer"
},
"price": {
"type": "double"
},
"totalAmount": {
"type": "double"
}
}
}
}
}
}
}
//聚合查询:分组,求最大,最小,平均值:
对一个数据集求最大、最小、和、平均值等指标的聚合,在ES中称为指标聚合(Metrics(指标))
Buckets(桶/集合):满足特定条件的文档的集合
public List<EsOrderMast> aggs(EsQueryObject queryObject) throws ExecutionException, InterruptedException {
TransportClient client = ESUtil.getClient();
SearchRequestBuilder sourceBuilder=client.prepareSearch("shop");
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
if(!StringUtils.isEmpty(queryObject.getOrderId())){
boolQueryBuilder.must(QueryBuilders.termQuery("orderId",queryObject.getOrderId()));
}
if(!StringUtils.isEmpty(queryObject.getCustomerId())){
boolQueryBuilder.must(QueryBuilders.termQuery("customerId",queryObject.getCustomerId()));
}
if(!StringUtils.isEmpty(queryObject.getTxtOrderTitle())){
boolQueryBuilder.must(QueryBuilders.matchQuery("orderTitle",queryObject.getTxtOrderTitle()));
} if(null!=queryObject.getOrderStatus() && queryObject.getOrderStatus()>=0){
boolQueryBuilder.must(QueryBuilders.termQuery("orderStatus",queryObject.getOrderStatus()));
}
if(null!=queryObject.getPaymentStatus() && queryObject.getPaymentStatus()>=0){
boolQueryBuilder.must(QueryBuilders.termQuery("paymentStatus",queryObject.getPaymentStatus()));
}
if(!StringUtils.isEmpty(queryObject.getPhone())){
boolQueryBuilder.must(QueryBuilders.termQuery("phone",queryObject.getPhone()));
}
if(!StringUtils.isEmpty(queryObject.getRecieveName())){
boolQueryBuilder.must(QueryBuilders.matchQuery("recieveName",queryObject.getRecieveName()));
}
if(!StringUtils.isEmpty(queryObject.getAddresss())){
boolQueryBuilder.must(QueryBuilders.matchQuery("addresss",queryObject.getAddresss()));
} if(!StringUtils.isEmpty(queryObject.getShopId())){
boolQueryBuilder.must(QueryBuilders.termQuery("shopId",queryObject.getShopId()));
}
if(!StringUtils.isEmpty(queryObject.getItemId())){
boolQueryBuilder.must(QueryBuilders.termQuery("itemList.itemId",queryObject.getItemId()));
}
if(!StringUtils.isEmpty(queryObject.getItemTile())){
boolQueryBuilder.must(QueryBuilders.matchQuery("itemList.itemTitle",queryObject.getItemTile()));
} if(!StringUtils.isEmpty(queryObject.getOrderSubmitTime_S())){
boolQueryBuilder.must(QueryBuilders.rangeQuery("orderSubmit").gte(queryObject.getOrderSubmitTime_S()));
}
if(!StringUtils.isEmpty(queryObject.getOrderSubmitTime_E())){
boolQueryBuilder.must(QueryBuilders.rangeQuery("orderSubmit").lte(queryObject.getOrderSubmitTime_E()));
}
if(!StringUtils.isEmpty(queryObject.getPayTime_S())){
boolQueryBuilder.must(QueryBuilders.rangeQuery("payTime").gte(queryObject.getPayTime_S()));
}
if(!StringUtils.isEmpty(queryObject.getPayTime_E())){
boolQueryBuilder.must(QueryBuilders.rangeQuery("payTime").lte(queryObject.getPayTime_E()));
} if(null!=queryObject.getMinPayAmount()){
boolQueryBuilder.must(QueryBuilders.rangeQuery("orderAmount").gte(queryObject.getMinPayAmount().doubleValue()));
}
if(null!=queryObject.getMaxPayAmount()){
boolQueryBuilder.must(QueryBuilders.rangeQuery("orderAmount").lte(queryObject.getMaxPayAmount().doubleValue()));
}
sourceBuilder.setQuery(boolQueryBuilder);
/*sourceBuilder.addSort(SortBuilders.fieldSort("orderSubmit").order(SortOrder.ASC));//排序
sourceBuilder.addAggregation(AggregationBuilders.max("max").field("orderAmount"));//最大值
sourceBuilder.addAggregation(AggregationBuilders.min("min").field("orderAmount")); //最小值
sourceBuilder.addAggregation(AggregationBuilders.count("total").field("orderId")); //总数
sourceBuilder.addAggregation(AggregationBuilders.sum("totalAmount").field("orderAmount"));//总金额
sourceBuilder.addAggregation(AggregationBuilders.avg("avg").field("orderAmount"));//平均金额*/
sourceBuilder.setSize(0);//设置为0就不用返回结果集,这样性能更高
//按商家分组再统计
TermsAggregationBuilder termsAggregationBuilder = AggregationBuilders.terms("shops").field("shopId").subAggregation(AggregationBuilders.max("max").field("orderAmount"))
.subAggregation(AggregationBuilders.avg("avg").field("orderAmount"));
sourceBuilder.addAggregation(termsAggregationBuilder);
ActionFuture<SearchResponse> execute = sourceBuilder.execute();
SearchResponse searchResponse = execute.get();
//Aggregation max = searchResponse.getAggregations().get("max");
// Aggregation min = searchResponse.getAggregations().get("min");
//Aggregation total = searchResponse.getAggregations().get("total");
//Aggregation totalAmount = searchResponse.getAggregations().get("totalAmount");
//System.out.println(max.toString());
// System.out.println(min.toString());
//System.out.println(total.toString());
// System.out.println(totalAmount.toString());
Aggregation shops = searchResponse.getAggregations().get("shops");
System.out.println(shops.toString());
//如果你要在商家分组之后继续分组,那只要加个subAggregation(AggregationBuilders.terms("xx").field("xx")) return null; }
//聚合如果报错,分组的字段要在,mapping中设置fildData属性为true,这里以商家名称
如:
"shopName": {
"type": "keyword",
"index": true,
"fielddata": true
}
官方文档介绍,type类型为text时,并且需要该字段进行分组时需要设置:
Fielddata is disabled on text fields by default. Set fielddata=true on [your_field_name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.
//自动补全介绍:在百度搜索时,经常会有自动补全功能,下面我以商品的标题作为补全字段
//将mysql商品表所有的商品标题存到ES中
//第一步创建mapping:
localhost:9200/shop_sugess
{
"mappings": {
"suggest": {
"properties": {
"titleSugess": {
"type": "completion",
"analyzer": "english",
"search_analyzer": "english"
}
}
}
}
}
//第二步初始化数据进es:
//批量新增
@Async
public void initSugess() throws IOException {
List<String> titleList = orderItemMapper.findItemTitleList();
TransportClient client = ESUtil.getClient();
BulkRequest bulkRequest = new BulkRequest();
if(null!=titleList && titleList.size()>0){
for (String title : titleList) {
IndexRequest indexRequest = new IndexRequest("shop_sugess", "suggest");
XContentBuilder endObject = XContentFactory.jsonBuilder().startObject()
.field("titleSugess",title)
.endObject();
indexRequest.source(endObject);
bulkRequest.add(indexRequest);
}
}
ActionFuture<BulkResponse> bulk = client.bulk(bulkRequest);
System.out.println(bulk.isDone());
BulkResponse bulkItemResponses = bulk.actionGet();
System.out.println(bulkItemResponses.status().getStatus());
}
//第三步开始查询:
//查询
public void testSugess(String prexfit){
String suggestField="title_suggest";//指定在哪个字段搜索
Integer suggestMaxCount=10;//获得最大suggest条数
CompletionSuggestionBuilder suggestionBuilderDistrict = new CompletionSuggestionBuilder("titleSugess").prefix(prexfit).size(suggestMaxCount);
SuggestBuilder suggestBuilder = new SuggestBuilder();
suggestBuilder.addSuggestion("my_suggest", suggestionBuilderDistrict);//添加suggest
TransportClient client = ESUtil.getClient();
SearchRequestBuilder requestBuilder =client.prepareSearch("shop_sugess").setTypes("suggest").suggest(suggestBuilder);
SearchResponse response = requestBuilder.get();
Suggest suggest = response.getSuggest();
Iterator<Suggest.Suggestion<? extends Suggest.Suggestion.Entry<? extends Suggest.Suggestion.Entry.Option>>> iterator = suggest.iterator();
while (iterator.hasNext()){
Suggest.Suggestion<? extends Suggest.Suggestion.Entry<? extends Suggest.Suggestion.Entry.Option>> next = iterator.next();
List<? extends Suggest.Suggestion.Entry<? extends Suggest.Suggestion.Entry.Option>> entries = next.getEntries();
for (Suggest.Suggestion.Entry<? extends Suggest.Suggestion.Entry.Option> entry : entries) {
List<? extends Suggest.Suggestion.Entry.Option> options = entry.getOptions();
for (Suggest.Suggestion.Entry.Option option : options) {
System.out.println(option.getText().toString());
}
}
}
}
//此处以Sam作为前缀查询:结果如下
//es批量新增的其他方式:
//批量新增
public void batchInsertHotWordSuggest(List<HotWorderSuggestDTO> list) {
if(CommonUtil.isNull(list)){
return;
}
BulkRequest bulkRequest = new BulkRequest();
list.stream().forEach(a->{
IndexRequest indexRequest = new IndexRequest(SUGGEST_INDEX_NAME, SUGGEST_TYPE_NAME, a.getId()); XContentBuilder suggestDTO = null;
try {
suggestDTO = XContentFactory.jsonBuilder()
.startObject()
.field("title_suggest", a.getTitle_suggest())
.endObject();
} catch (IOException e) {
logger.info("热搜词同步失败...."+e.getMessage()+a.getTitle_suggest());
}
if(null!=suggestDTO){
indexRequest.source(suggestDTO);
bulkRequest.add(indexRequest);
}
});
ActionFuture<BulkResponse> bulk = client.bulk(bulkRequest);
if(null!=bulk.actionGet() && null!=bulk.actionGet().status() && bulk.actionGet().status().getStatus()!=200){
logger.info("批量同步搜索词失败");
} }
public List<String> findShopItemList(RequestDTO requestDTO, Page page) { SearchRequest searchRequest = new SearchRequest(INDEX_NAME);
searchRequest.types(TYPE_NAME);
// 构造查询器
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); //品牌
if (StringUtils.isNotBlank(requestDTO.getBrandId())) {
boolQueryBuilder.must(QueryBuilders.termsQuery("brandId", requestDTO.getBrandId()));
}
//分类
if (StringUtils.isNotBlank(requestDTO.getCategoryId())) {//前缀查询
boolQueryBuilder.must(QueryBuilders.prefixQuery("classId", requestDTO.getCategoryId()));
} boolQueryBuilder.must(QueryBuilders.termQuery("flgItemOnline", ItemTypeEnum.FLG_ITEM_TYPE_ONE.getValue().toString()));
//or查询
boolQueryBuilder.must(QueryBuilders.boolQuery().should(QueryBuilders.termQuery("bolStock", "y")).should(QueryBuilders.termQuery("bolOverdue", "y")));
boolQueryBuilder.must(QueryBuilders.boolQuery().must(QueryBuilders.rangeQuery("datSellStart").lte(DateUtil.getTime())).must(QueryBuilders.rangeQuery("datSellEnd").gte(DateUtil.getTime())));
sourceBuilder.query(boolQueryBuilder);
sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
if(null==page){
page=new Page(1,10);
}
if(page.getPageSize()<=0){
page.setPageSize(10);
} sourceBuilder.from((page.getPageNum() == 0 ? 0 : page.getPageNum() - 1) * page.getPageSize());
sourceBuilder.size(page.getPageSize());
Integer sortPrice = requestDTO.getSortPrice();
Integer sortSale = requestDTO.getSortSale();
if(null!=sortSale && sortSale>0 ){// "排序(0,不排序 1.降序 2.升序)
sourceBuilder.sort(new FieldSortBuilder("sellTotal").order(sortSale==1?SortOrder.DESC:SortOrder.ASC));
}
if(null!=sortPrice && sortPrice>0 ) {// 排序排行(0,不排序 1.降序 2.升序)
sourceBuilder.sort(new FieldSortBuilder("price").order(sortSale == 1 ? SortOrder.DESC : SortOrder.ASC));
} searchRequest.source(sourceBuilder);
searchRequest.searchType(SearchType.QUERY_THEN_FETCH);
SearchResponse searchResponse = client.search(searchRequest).actionGet(); if(null==searchResponse || null==searchResponse.getHits() || searchResponse.getHits().totalHits<=0){
return null;
}
List<String> list = new ArrayList<>();
SearchHits hits = searchResponse.getHits();
for (SearchHit hit : hits) {
Map<String, Object> hitMap = hit.getSourceAsMap();
if (null != hitMap) {
String itemId = (String) hitMap.get("itemId");
list.add(itemId);
}
}
return list;
}
java整合Elasticsearch,实现crud以及高级查询的分页,范围,排序功能,泰文分词器的使用,分组,最大,最小,平均值,以及自动补全功能的更多相关文章
- StringBoot整合ELK实现日志收集和搜索自动补全功能(详细图文教程)
@ 目录 StringBoot整合ELK实现日志收集和搜索自动补全功能(详细图文教程) 一.下载ELK的安装包上传并解压 1.Elasticsearch下载 2.Logstash下载 3.Kibana ...
- 第三百六十八节,Python分布式爬虫打造搜索引擎Scrapy精讲—elasticsearch(搜索引擎)用Django实现搜索的自动补全功能
第三百六十八节,Python分布式爬虫打造搜索引擎Scrapy精讲—用Django实现搜索的自动补全功能 elasticsearch(搜索引擎)提供了自动补全接口 官方说明:https://www.e ...
- 四十七 Python分布式爬虫打造搜索引擎Scrapy精讲—elasticsearch(搜索引擎)用Django实现搜索的自动补全功能
elasticsearch(搜索引擎)提供了自动补全接口 官方说明:https://www.elastic.co/guide/en/elasticsearch/reference/current/se ...
- 使用插件bootstrap-table实现表格记录的查询、分页、排序等处理
在业务系统开发中,对表格记录的查询.分页.排序等处理是非常常见的,在Web开发中,可以采用很多功能强大的插件来满足要求,且能极大的提高开发效率,本随笔介绍这个bootstrap-table是一款非常有 ...
- JAVA学习笔记(1)—— eclipse自动补全和主题及字体配置
1.自动补全功能 (1)打开 Eclipse -> Window -> Perferences (2)选择Java -> Editor -> Content Assist -& ...
- 基于Metronic的Bootstrap开发框架经验总结(16)-- 使用插件bootstrap-table实现表格记录的查询、分页、排序等处理
在业务系统开发中,对表格记录的查询.分页.排序等处理是非常常见的,在Web开发中,可以采用很多功能强大的插件来满足要求,且能极大的提高开发效率,本随笔介绍这个bootstrap-table是一款非常有 ...
- 测试使用索引库crud和高级查询分页
1.搭建ES的服务 导入依赖 <dependency> <groupId>org.springframework.boot</groupId> <artifa ...
- Linq高级查询,分页查询及查询分页结合
一.高级查询与分页查询 1.以...开头 StartsWith Repeater1.DataSource=con.Users.Where(r=>r.Nickname.StartsWith( ...
- ES系列十三、Elasticsearch Suggester API(自动补全)
1.概念 1.补全api主要分为四类 Term Suggester(纠错补全,输入错误的情况下补全正确的单词) Phrase Suggester(自动补全短语,输入一个单词补全整个短语) Comple ...
随机推荐
- OpenGL学习日志(2020.4之前)
咳咳,原本这个日志是本机上随便写的一些记录,也没怎么注意可读性和格式,有用信息密度很小,所以实用价值并不大.暂时由于不可抗因素得先鸽一段落了... 后续的日志会升格为模块化的学习记录,(应该)将会有很 ...
- 《HelloGitHub》第 53 期
兴趣是最好的老师,HelloGitHub 就是帮你找到兴趣! 简介 分享 GitHub 上有趣.入门级的开源项目. 这是一个面向编程新手.热爱编程.对开源社区感兴趣 人群的月刊,月刊的内容包括:各种编 ...
- jmeter http并发测试时报错
错误信息如下:jmeter Response code: Non HTTP response code: java.net.URISyntaxException 网上收了一大堆,都没法解决 我的用到了 ...
- android 数据绑定(5) kotlin 的binding bug
1.BR找不到,无法自动更新 1.1 描述 https://stackoverflow.com/questions/57983508/android-studio-kotlin-databinding ...
- iOS NSNotification通知
通知中心(NSNotificationCenter) 通知(NSNotification) 一个完整的通知一般包含3个属性:(注意顺序) - (NSString *)name; 通知的名称 - (i ...
- Fitness - 06.01
倒计时213天 久违的瑜伽课,却发现生疏了很多,倒地不起TAT 要加强锻炼,不要松懈啊~~~! 期待黄金周的到来!!
- MySQL 增删查改 必知必会
MySQL 数据库中的基础操作 3.表的修改 对表的表名.字段.字段类型.字段长度.约束等进行修改. 3.1 表的名称修改 -- 语法: ALTER TABLE 库名.表名 RENAME TO 新表名 ...
- 当年偶然发现的 Java Bug(JDK 9及之前仍未修复)
背景 15年在中信银行做持续集成时,由于当时的项目是基于三方采购的 Java 配置开发平台做的,平台自己基于 Ant 插件实现了增量和热部署. 其中有几个项目在持续集成部署时,经常发现 Linux 平 ...
- 创建DBA用户luna
用system/pswd登陆sql plus,执行下面命令: 请输入用户名: system 输入口令: 连接到: Oracle Database 11g Enterprise Edition Rele ...
- python之ddt模块使用
一.DDT(数据驱动)简介 Data-Driven Tests(DDT)即数据驱动测试,可以实现不同数据运行同一个测试用例(通过数据的不同来驱动测试结果的不同). ddt本质其实就是装饰器,一组数据一 ...