03-springboot整合elasticsearch-源码初识
前面两个小节已经知道了spring boot怎么整合es,以及es的简单使用,但是springboot中是怎么和es服务器交互的。我们可以简单了解一下。要看一下源码
在看源码的同时,先要对springboot请求ES服务器的原理了解一下,ES官网(https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-update.html)给出了很详细的说明,可以自行进行了解。
1.RestClient es交互的基础服务器
对于单机es,一般使用的是ElasticsearchOperations
1.1 数据存储的具体过程
本质上还是使用ElasticsearchRestTemplate进行操作。
数据存储发起操作
1.存储数据
String documentId = operations.index(indexQuery,indexCoordinates);
2.index操作的过程
@Override
public String index(IndexQuery query, IndexCoordinates index) {
maybeCallbackBeforeConvertWithQuery(query, index);//实体类再保存操作之前的回调方法
IndexRequest request = requestFactory.indexRequest(query, index);//==获取indexRequest的过程==
String documentId = execute(client -> client.index(request, RequestOptions.DEFAULT).getId());//==与ES服务器交互过程==
// We should call this because we are not going through a mapper.
Object queryObject = query.getObject();
if (queryObject != null) {
setPersistentEntityId(queryObject, documentId);
}
maybeCallbackAfterSaveWithQuery(query, index);//实体类再保存之后的回调方法
return documentId;
}
查看IndexRequest的创建过程如下
public IndexRequest indexRequest(IndexQuery query, IndexCoordinates index) {
String indexName = index.getIndexName();
IndexRequest indexRequest;
if (query.getObject() != null) {
String id = StringUtils.isEmpty(query.getId()) ? getPersistentEntityId(query.getObject()) : query.getId();
// If we have a query id and a document id, do not ask ES to generate one.
if (id != null) {
indexRequest = new IndexRequest(indexName).id(id);
} else {
indexRequest = new IndexRequest(indexName);
}
/**
* 1.将传来的object转成Map,再转成json串
* 2.将Object的json串转成字节BytesReference,请求的ContentType设置为Request.JSON方式
*/
indexRequest.source(elasticsearchConverter.mapObject(query.getObject()).toJson(), Requests.INDEX_CONTENT_TYPE);
}
// 省略一部分代码。。。。。
return indexRequest;
}
请求数据处理过程
public final IndexResponse index(IndexRequest indexRequest, RequestOptions options) throws IOException {
return performRequestAndParseEntity(indexRequest, RequestConverters::index, options, IndexResponse::fromXContent, emptySet());
}
1.RequestConverters::index 这个过程创建Request对象
static Request index(IndexRequest indexRequest) {
String method = Strings.hasLength(indexRequest.id()) ? HttpPut.METHOD_NAME : HttpPost.METHOD_NAME; //根据有无ID选择传输方式是PUT还是POST
boolean isCreate = (indexRequest.opType() == DocWriteRequest.OpType.CREATE);
//拼接请求的uri
String endpoint = endpoint(indexRequest.index(), indexRequest.type(), indexRequest.id(), isCreate ? "_create" : null);
Request request = new Request(method, endpoint);
//增加request的请求参数
Params parameters = new Params(request);
parameters.withRouting(indexRequest.routing());
parameters.withParent(indexRequest.parent());
parameters.withTimeout(indexRequest.timeout());
parameters.withVersion(indexRequest.version());
parameters.withVersionType(indexRequest.versionType());
parameters.withIfSeqNo(indexRequest.ifSeqNo());
parameters.withIfPrimaryTerm(indexRequest.ifPrimaryTerm());
parameters.withPipeline(indexRequest.getPipeline());
parameters.withRefreshPolicy(indexRequest.getRefreshPolicy());
parameters.withWaitForActiveShards(indexRequest.waitForActiveShards(), ActiveShardCount.DEFAULT);
//将请求的参数变成byte[]
BytesRef source = indexRequest.source().toBytesRef();
ContentType contentType = createContentType(indexRequest.getContentType());
request.setEntity(new NByteArrayEntity(source.bytes, source.offset, source.length, contentType));
return request;
}
2.创建response
3.创建空的集合 : new EmptySet<>();
4. 给ES服务器发送数据
private <Req, Resp> Resp internalPerformRequest(Req request,
CheckedFunction<Req, Request, IOException> requestConverter,
RequestOptions options,
CheckedFunction<Response, Resp, IOException> responseConverter,
Set<Integer> ignores) throws IOException {
Request req = requestConverter.apply(request);
req.setOptions(options);
Response response;
try {
response = client.performRequest(req);
} catch (ResponseException e) {
if (ignores.contains(e.getResponse().getStatusLine().getStatusCode())) {
try {
return responseConverter.apply(e.getResponse());
} catch (Exception innerException) {
throw parseResponseException(e);
}
}
throw parseResponseException(e);
}
try {
return responseConverter.apply(response);
} catch(Exception e) {
throw new IOException("Unable to parse response body for " + response, e);
}
}
RestClien请求发送的过程
1.发送请求过程
public Response performRequest(Request request) throws IOException {
SyncResponseListener listener = new SyncResponseListener(maxRetryTimeoutMillis);
performRequestAsyncNoCatch(request, listener);
return listener.get();
}
//创建请求的url,创建request对象
void performRequestAsyncNoCatch(Request request, ResponseListener listener) throws IOException {
Map<String, String> requestParams = new HashMap<>(request.getParameters());
String ignoreString = requestParams.remove("ignore");
Set<Integer> ignoreErrorCodes;
if (ignoreString == null) {
if (HttpHead.METHOD_NAME.equals(request.getMethod())) {
//404 never causes error if returned for a HEAD request
ignoreErrorCodes = Collections.singleton(404);
} else {
ignoreErrorCodes = Collections.emptySet();
}
} else {
String[] ignoresArray = ignoreString.split(",");
ignoreErrorCodes = new HashSet<>();
if (HttpHead.METHOD_NAME.equals(request.getMethod())) {
//404 never causes error if returned for a HEAD request
ignoreErrorCodes.add(404);
}
for (String ignoreCode : ignoresArray) {
try {
ignoreErrorCodes.add(Integer.valueOf(ignoreCode));
} catch (NumberFormatException e) {
throw new IllegalArgumentException("ignore value should be a number, found [" + ignoreString + "] instead", e);
}
}
}
URI uri = buildUri(pathPrefix, request.getEndpoint(), requestParams);//创建url和请求参数拼接
HttpRequestBase httpRequest = createHttpRequest(request.getMethod(), uri, request.getEntity());//创建request对象
setHeaders(httpRequest, request.getOptions().getHeaders());
FailureTrackingResponseListener failureTrackingResponseListener = new FailureTrackingResponseListener(listener);
long startTime = System.nanoTime();
performRequestAsync(startTime, nextNode(), httpRequest, ignoreErrorCodes,
request.getOptions().getWarningsHandler() == null ? warningsHandler : request.getOptions().getWarningsHandler(),
request.getOptions().getHttpAsyncResponseConsumerFactory(), failureTrackingResponseListener);//发送过程
}
//数据发送,完成后将响应信息封装进response对象中
private void performRequestAsync(final long startTime, final NodeTuple<Iterator<Node>> nodeTuple, final HttpRequestBase request,
final Set<Integer> ignoreErrorCodes,
final WarningsHandler thisWarningsHandler,
final HttpAsyncResponseConsumerFactory httpAsyncResponseConsumerFactory,
final FailureTrackingResponseListener listener) {
final Node node = nodeTuple.nodes.next(); //获取注册的节点
final HttpAsyncRequestProducer requestProducer = HttpAsyncMethods.create(node.getHost(), request);
final HttpAsyncResponseConsumer<HttpResponse> asyncResponseConsumer =
httpAsyncResponseConsumerFactory.createHttpAsyncResponseConsumer();
final HttpClientContext context = HttpClientContext.create();
context.setAuthCache(nodeTuple.authCache);
client.execute(requestProducer, asyncResponseConsumer, context, new FutureCallback<HttpResponse>() {
@Override
public void completed(HttpResponse httpResponse) { //执行完成后的回调方法
try {
RequestLogger.logResponse(logger, request, node.getHost(), httpResponse);
int statusCode = httpResponse.getStatusLine().getStatusCode();
Response response = new Response(request.getRequestLine(), node.getHost(), httpResponse);
if (isSuccessfulResponse(statusCode) || ignoreErrorCodes.contains(response.getStatusLine().getStatusCode())) {
onResponse(node);
if (thisWarningsHandler.warningsShouldFailRequest(response.getWarnings())) {
listener.onDefinitiveFailure(new WarningFailureException(response));
} else {
listener.onSuccess(response);
}
} else {
ResponseException responseException = new ResponseException(response);
if (isRetryStatus(statusCode)) {
//mark host dead and retry against next one
onFailure(node);
retryIfPossible(responseException);
} else {
//mark host alive and don't retry, as the error should be a request problem
onResponse(node);
listener.onDefinitiveFailure(responseException);
}
}
} catch(Exception e) {
listener.onDefinitiveFailure(e);
}
}
});
}
03-springboot整合elasticsearch-源码初识的更多相关文章
- springboot整合mybatis源码分析
springboot整合mybatis源码分析 本文主要讲述mybatis在springboot中是如何被加载执行的,由于涉及的内容会比较多,所以这次只会对调用关系及关键代码点进行讲解,为了避免文章太 ...
- tomcat源码--springboot整合tomcat源码分析
1.测试代码,一个简单的springboot web项目:地址:https://gitee.com/yangxioahui/demo_mybatis.git 一:tomcat的主要架构:1.如果我们下 ...
- SpringBoot整合Elasticsearch详细步骤以及代码示例(附源码)
准备工作 环境准备 JAVA版本 java version "1.8.0_121" Java(TM) SE Runtime Environment (build 1.8.0_121 ...
- Elasticsearch源码分析 - 源码构建
原文地址:https://mp.weixin.qq.com/s?__biz=MzU2Njg5Nzk0NQ==&mid=2247483694&idx=1&sn=bd03afe5a ...
- 渣渣菜鸡的 ElasticSearch 源码解析 —— 环境搭建
关注我 转载请务必注明原创地址为:http://www.54tianzhisheng.cn/2018/08/25/es-code01/ 软件环境 1.Intellij Idea:2018.2版本 2. ...
- Java 集合系列 03 ArrayList详细介绍(源码解析)和使用示例
java 集合系列目录: Java 集合系列 01 总体框架 Java 集合系列 02 Collection架构 Java 集合系列 03 ArrayList详细介绍(源码解析)和使用示例 Java ...
- Springboot security cas源码陶冶-ExceptionTranslationFilter
拦截关键的两个异常,对异常进行处理.主要应用异常则跳转至cas服务端登录页面 ExceptionTranslationFilter#doFilter-逻辑入口 具体操作逻辑如下 public void ...
- SpringBoot整合ElasticSearch实现多版本的兼容
前言 在上一篇学习SpringBoot中,整合了Mybatis.Druid和PageHelper并实现了多数据源的操作.本篇主要是介绍和使用目前最火的搜索引擎ElastiSearch,并和Spring ...
- ElasticSearch(2)---SpringBoot整合ElasticSearch
SpringBoot整合ElasticSearch 一.基于spring-boot-starter-data-elasticsearch整合 开发环境:springboot版本:2.0.1,elast ...
- SpringBoot自动配置源码调试
之前对SpringBoot的自动配置原理进行了较为详细的介绍(https://www.cnblogs.com/stm32stm32/p/10560933.html),接下来就对自动配置进行源码调试,探 ...
随机推荐
- Hive的基本介绍以及常用函数
一.Hive的简介: Hive是基于Hadoop的一个数据仓库工具,可以将结构化的数据文件映射为一张表,并提供类SQL查询功能. 优点: ) 操作接口采用类SQL语法,提供快速开发的能力(简单.容易上 ...
- [OpenGL](翻译+补充)投影矩阵的推导
1.简介 基本是翻译和补充 http://www.songho.ca/opengl/gl_projectionmatrix.html 计算机显示器是一个2D的平面,一个3D的场景要被OpenGL渲染必 ...
- 温故知新-java虚拟机
文章目录 java虚拟机是什么? jvm的体系结构 第一个类加载子系统 类的生命周期 加载器分类 类加载机制 第二个运行时数据区(内存结构) GC算法和收集器 如何判断对象可以被回收? 如何判断一个常 ...
- 括号匹配-算法详细题解LeetCode
题目:有效的括号 给定一个只包括 '(',')','{','}','[',']' 的字符串,判断字符串是否有效. 有效字符串需满足: 左括号必须用相同类型的右括号闭合. 左括号必须以正确的顺序闭合. ...
- 实验五 shell脚本编程
项目 内容 这个作业属于哪个课程 课程链接 这个作业的要求在哪里 作业要求 学号-姓名 17041428-朱槐健 作业学习目标 1. 了解shell脚本的概念及使用 2.掌握shell脚本语言的基本语 ...
- ODEINT 求解常微分方程(4)
import numpy as np from scipy.integrate import odeint import matplotlib.pyplot as plt # function tha ...
- 存储过程WHERE条件不生效
业务上有个删除操作需要涉及到几张表,一条一条操作很麻烦,所以想写个存储过程来封装下,原始语句如下: DELETE FROM AUTH_AUTHORITY WHERE `ID` in ('f_view_ ...
- uni-app动态修改顶部导航栏标题
动态修改顶部导航栏标题有两种方法方式一.使用自定义到导航栏,覆盖原生导航栏 缺点:自定义到导航栏性能远远不如原生导航栏,手机顶部状态栏区域会被页面内容覆盖,这是因为窗体是沉浸式的原因,即全屏可写内容: ...
- [每日一题2020.06.12]P3375 【模板】KMP字符串匹配
题目链接 关于kmp : https://www.cnblogs.com/roccoshi/p/13096988.html 关于kmp, 想了很久, 我觉得不应该放在这里写, 另开一贴记录一下. #i ...
- LR字符串处理函数-lr_save_var
int lr_save_var ("截取的字符串"+start,len,options,param_name) start:表示从第几位截取 len:表示截取长度 option ...