ElasticSearch+Kibana 索引操作( 附源码)

一 前言

ElasticiSearch 简介 ElasticSearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用Java开发的，并作为Apache许可条款下的开放源码发布，是当前流行的企业级搜索引擎。设计用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。

因为在工作的项目中有使用到所以写下相关的内容，并附带源码 感兴趣的朋友可以自己玩一玩，整个项目都是在Linux上跑的，所以安装步骤都以Linux为主。什么？你不会Linux？ 学啊...........

二  安装步骤

2.1 Elasticsearch2.2安装（Yum安装步骤）

下载安装签名

rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

配置更新源

在 /etc/yum.repos.d/ 目录下创建.repo 后缀的文件,如elasticsearch.repo，编辑内容如下：

[elasticsearch-2.x]
name=Elasticsearch repository for2.x packages
baseurl=https://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

使用yum命令安装：

yum install elasticsearch

加入启动项：

chkconfig --add elasticsearch

基本配置

编辑文件/etc/elasticsearch/elasticsearch.yml，根据实际情况配置如下节点(也可使用默认)

• cluster.name
• node.name
• index.number_of_shards
• index.number_of_replicas
• network.host

启动Es

运行命令：

Service ElasticSearch start

参考地址：

• https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html

---

2.2 Kibana安装

Kibana 可使用它对数据进行高效的搜索、可视化、分析等各种操作

下载解压安装包

下载地址：

• https://www.elastic.co/downloads/kibana

配置

编辑文件config/kibana.yml ，配置属性：

• server.host (站点地址)
• elasticsearch.url (指向Elasticsearch 实例)

启动

.bin/kibana

参考地址：

• https://www.elastic.co/downloads/kibana

---

2.3 Marvel安装

Marvel是Elasticsearch的管理和监控工具

安装Marvel代理插件到各个ES节点

在ES目录下运行 bin/plugin 安装插件许可

bin/plugin install license

运行bin/plugin install 安装Marvel代理插件

bin/plugin install marvel-agent

将Marvel插件安装到Kiabana

运行命令：

bin/kibana plugin --install elasticsearch/marvel/latest

重启 Kibana

参考地址：

• https://www.elastic.co/downloads/marvel
• https://www.elastic.co/guide/en/marvel/current/getting-started.html

---

2.4 Sense安装

一个flask写的elasticsearch查询工具:

支持es查询语言自动提示，es结构自动提示，支持两种主题，支持查询历史记录，支持快捷键。

到Kibana目录运行命令安装 Sense插件

./bin/kibana plugin --install elastic/sense

重新启动 Kibana

参考地址：

• https://www.elastic.co/guide/en/sense/master/installing.html

2.5 安装Ik分词插件

获取源码

git clone https://github.com/medcl/elasticsearch-analysis-ik.git

编译

去源码根目录下编译

cd elasticsearch-analysis-ik
mvn clean
mvn compile
mvn package

将文件 #{project_path}/elasticsearch-analysis-ik/target/releases/elasticsearch-analysis-ik-*.zip 复制解压到 elasticsearch的目录: plugins/ik

这几个插件安装完成后效果如下

三 ElasticSearch客户端操作索引

.net 版的客户端 是NEST 语法有相应的官网文档解释很全，也很简单 http://nest.azurewebsites.net/nest/search/basics.html

示例代码只写了最常用的对索引的一些基本操作 新建， 删除， 添加数据， 局部更新，给索引设别名

 public class ElasticSearchService
     {
         private IElasticClient EsClient;
         public ElasticSearchService()
         {
             EsClient = ElasticSearchConfig.EsClient;
         }

         /// <summary>
         /// 创建索引并设置别名
         /// </summary>
         /// <returns></returns>
         public string CreateIndex()
         {
             string newIndex = ElasticSearchConfig.IndexName + DateTime.Now.ToString("yyyyMMddHHmm");
             //创建索引，设置mapping
             var response = EsClient.CreateIndex(newIndex, c =>
                 c.Mappings(m => m.Map<DoctorEntity>(map => map.AutoMap()))
                 .Settings(setting=>setting.NumberOfShards().NumberOfReplicas()));//3个分片，1个副本
             if (!response.Acknowledged)
             {
                 return "创建索引失败" + response.ServerError.Error.Reason;
             }
             //设置别名
             var aliasResponse = EsClient.Alias(new BulkAliasDescriptor().Add(c => c.Index(newIndex).Alias(ElasticSearchConfig.IndexName)));
             if (!aliasResponse.Acknowledged)
             {
                 return "设置索引别名失败" + aliasResponse.ServerError.Error.Reason;
             }

             return "创建索引并设置别名成功！";
         }

         /// <summary>
         /// 移除别名与索引
         /// </summary>
         /// <returns></returns>
         public string DeleteIndex()
         {
             //获取别名下的索引
             var alias = EsClient.CatAliases(a => a.Name(ElasticSearchConfig.IndexName));
             BulkAliasDescriptor aliasBulk = new BulkAliasDescriptor();
             var indexNames = new List<string>();
             foreach (var record in alias.Records)
             {
                 //移除别名下的索引
                 aliasBulk.Add(new AliasRemoveDescriptor().Index(record.Index).Alias(record.Alias));
                 indexNames.Add(record.Index);
             }
             var response = EsClient.Alias(aliasBulk);

             //删除旧索引
             indexNames.ForEach(index => EsClient.DeleteIndex(new DeleteIndexRequest(index)));

             if (!response.Acknowledged&&alias.Records.Any())
             {
                 return "索引与别名关系移除失败！" + response.ServerError.Error.Reason;
             }

             return "索引与别名关系移除成功！";
         }

         /// <summary>
         /// 切换索引别名
         /// </summary>
         public string ChangeIndexAliase(string newIndexName)
         {
             //1 创建新的索引后，将别名指向的索引改为新索引名
             var alias = EsClient.CatAliases(new CatAliasesRequest(ElasticSearchConfig.IndexName));

             BulkAliasDescriptor aliasDescriptor = new BulkAliasDescriptor();

             foreach (var record in alias.Records)
             {
                 aliasDescriptor.Add(new AliasRemoveDescriptor().Index(record.Index).Alias(record.Alias));
             }

             aliasDescriptor.Add(new AliasAddDescriptor().Alias(ElasticSearchConfig.IndexName).Index(newIndexName));
             var response = EsClient.Alias(aliasDescriptor);
             if (!response.Acknowledged)
             {
                 return "切换索引别名失败" + response.ServerError.Error.Reason;
             }

             return string.Empty;
         }

         /// <summary>
         /// 向索引中添加数据
         /// </summary>
         /// <param name="doctorEntities"></param>
         /// <returns></returns>
         public string AddDoctorInfoToIndex(List<DoctorEntity> doctorEntities)
         {

             BulkRequest bulk = new BulkRequest(ElasticSearchConfig.IndexName)
             {
                 Operations = new List<IBulkOperation>()
             };
             foreach (var doctorEntity in doctorEntities)
             {
                 bulk.Operations.Add(new BulkIndexOperation<DoctorEntity>(doctorEntity));
             }

             var response = EsClient.Bulk(bulk);
             if (response.Errors)
             {
                 return "添加索引数据失败" + response.Items.First().Error;
             }

             return "添加索引数据成功！";
         }

         /// <summary>
         /// 更新索引数据
         /// </summary>
         public string UpdateDoctorInfoToIndex(List<DoctorEntity> doctorEntities)
         {
             var bulk = new BulkRequest(ElasticSearchConfig.IndexName)
             {
                 Operations = new List<IBulkOperation>()
             };
             BulkUpdateDescriptor<DoctorEntity, PartialDoctorEntity> updateDescriptor = new BulkUpdateDescriptor<DoctorEntity, PartialDoctorEntity>();

             foreach (var doctorEntity in doctorEntities)
             {
                var updatedescript =  updateDescriptor.IdFrom(doctorEntity)//会自动推断出document的id
                     .Doc(PartialDoctorEntity.Generate(doctorEntity))
                     .Upsert(doctorEntity)
                     .RetriesOnConflict();

                bulk.Operations.Add(updatedescript);
             }

             var response = EsClient.Bulk(bulk);

             if (response.Errors)
             {
                 return "更新索引数据失败" + response.ItemsWithErrors;
             }

             return "更新索引数据成功！";
         }

         /// <summary>
         /// 删除索引中数据
         /// </summary>
         /// <param name="doctors"></param>
         /// <returns></returns>
         public string DeleteDoctorInfoToIndex(List<DoctorEntity> doctors)
         {

             BulkRequest bulk = new BulkRequest(ElasticSearchConfig.IndexName)
             {
                 Operations = new List<IBulkOperation>()
             };

             foreach (var doctor in doctors)
             {
                 var deleteDescript = new BulkDeleteDescriptor<DoctorEntity>().Document(doctor);
                 bulk.Operations.Add(deleteDescript);
             }

             var response =  EsClient.Bulk(bulk);
             if (response.Errors)
             {
                 return "删除索引数据失败" + response.Items.First().Error;
             }

             return "删除索引数据成功";
         }

         /// <summary>
         /// 查询索引数据
         /// </summary>
         /// <param name="doctorId"></param>
         /// <returns></returns>
         public List<DoctorEntity> QueryDoctors(string doctorId)
         {
             SearchDescriptor<DoctorEntity> searchDescriptor =new SearchDescriptor<DoctorEntity>();

             searchDescriptor.Query(q => q.Term(t => t.Field("doctorId").Value(doctorId)));

             var result = EsClient.Search<DoctorEntity>(searchDescriptor);

             return result.Documents.ToList();
         }

         public List<DoctorEntity> BuiDoctorEntities()
         {
             var doctorList = new List<DoctorEntity>();
             string[] doctorNames = new string[] { "石霖", "陆桂香", "蔡江云", "刘玉凤", "谭志团", "贾雁平", "周琼华", "张平", "周华", "赵子龙" };
             for (int i = ; i < ; i++)
             {
                 doctorList.Add(new DoctorEntity()
                 {
                     DoctorId = "" + i,
                     DoctorName = doctorNames[i],
                     DoctorNumber = "" + i,
                     DepartmentNumber = "",
                     HospitalDepartmentId = Guid.NewGuid().ToString(),
                     HospitalDepartmentName = "内科",
                     HospitalId = Guid.NewGuid().ToString(),
                     HospitalName = "北京大学深圳医院",
                     HospitalNumber = "bjdxszyy",
                     ProfessionalDepartmentId = "" + i,
                     ProfessionalDepartmentName = "心胸内科",
                     SupplierNumber = "Thirdpart"
                 });
             }

             return doctorList;
         }

     }

示例代码

四 总结

　　以上只是对ElasticSearch 最简单基本的一些操作与概念，大家如果有兴趣可以继续深入了解。

GitHub 源码下载：https://github.com/ZeryZhang/ElasticSearchDemo

参考资料: http://www.linuxidc.com/Linux/2015-02/114243.htm

https://www.elastic.co/

http://nest.azurewebsites.net/nest/search/basics.html