Elasticsearch初探（一）

一、安装（Linux篇）

现在的开源软件越来越成熟，都打好包了，下载后直接解压就可以测试了。

 curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz
 tar -zxvf elasticsearch-6.3..tar.gz
 cd elasticsearch-6.3./bin/
 ./elasticsearch

注意：elasticsearch不允许使用root权限启动，所以使用root下载解压的同学，要chown改变一下文件夹的用户权限。

看到如下信息就说明启动成功了。

[--12T11::,][INFO ][o.e.n.Node               ] [] initializing ...
[--12T11::,][INFO ][o.e.e.NodeEnvironment    ] [4LdtI3q] using [] data paths, mounts [[/home (/dev/mapper/cl-home)]], net usable_space [.9gb], net total_space [.9gb], types [xfs]
[--12T11::,][INFO ][o.e.e.NodeEnvironment    ] [4LdtI3q] heap size [.7mb], compressed ordinary object pointers [true]
[--12T11::,][INFO ][o.e.n.Node               ] [4LdtI3q] node name derived from node ID [4LdtI3q-T2ehl0Z6Xv6cOA]; set [node.name] to override
[--12T11::,][INFO ][o.e.n.Node               ] [4LdtI3q] version[6.3.], pid[], build[default/tar/eb782d0/--29T21::.107521Z], OS[Linux/3.10.-.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) -Bit Server VM/1.8.0_112/25.112-b15]
[--12T11::,][INFO ][o.e.n.Node               ] [4LdtI3q] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.af3cuwkJ, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=, -XX:GCLogFileSize=64m, -Des.path.home=/home/lings/elastic/elasticsearch-6.3., -Des.path.conf=/home/lings/elastic/elasticsearch-6.3./config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [aggs-matrix-stats]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [analysis-common]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [ingest-common]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [lang-expression]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [lang-mustache]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [lang-painless]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [mapper-extras]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [parent-join]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [percolator]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [rank-eval]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [reindex]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [repository-url]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [transport-netty4]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [tribe]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-core]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-deprecation]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-graph]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-logstash]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-ml]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-monitoring]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-rollup]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-security]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-sql]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-upgrade]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] loaded module [x-pack-watcher]
[--12T11::,][INFO ][o.e.p.PluginsService     ] [4LdtI3q] no plugins loaded
[--12T11::,][INFO ][o.e.x.s.a.s.FileRolesStore] [4LdtI3q] parsed [] roles from file [/home/lings/elastic/elasticsearch-6.3./config/roles.yml]
[--12T11::,][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/] [Main.cc@] controller ( bit): Version 6.3. (Build 4d0b8f0a0ef401) Copyright (c)  Elasticsearch BV
[--12T11::,][DEBUG][o.e.a.ActionModule       ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[--12T11::,][INFO ][o.e.d.DiscoveryModule    ] [4LdtI3q] using discovery type [zen]
[--12T11::,][INFO ][o.e.n.Node               ] [4LdtI3q] initialized
[--12T11::,][INFO ][o.e.n.Node               ] [4LdtI3q] starting ...
[--12T11::,][INFO ][o.e.t.TransportService   ] [4LdtI3q] publish_address {127.0.0.1:}, bound_addresses {[::]:}, {127.0.0.1:}
[--12T11::,][WARN ][o.e.b.BootstrapChecks    ] [4LdtI3q] max file descriptors [] for elasticsearch process is too low, increase to at least []
[--12T11::,][WARN ][o.e.b.BootstrapChecks    ] [4LdtI3q] max virtual memory areas vm.max_map_count [] is too low, increase to at least []
[--12T11::,][INFO ][o.e.c.s.MasterService    ] [4LdtI3q] zen-disco-elected-as-master ([] nodes joined)[, ], reason: new_master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:}{ml.machine_memory=, xpack.installed=true, ml.max_open_jobs=, ml.enabled=true}
[--12T11::,][INFO ][o.e.c.s.ClusterApplierService] [4LdtI3q] new_master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:}{ml.machine_memory=, xpack.installed=true, ml.max_open_jobs=, ml.enabled=true}, reason: apply cluster state (from master [master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:}{ml.machine_memory=, xpack.installed=true, ml.max_open_jobs=, ml.enabled=true} committed version [] source [zen-disco-elected-as-master ([] nodes joined)[, ]]])
[--12T11::,][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [4LdtI3q] publish_address {127.0.0.1:}, bound_addresses {[::]:}, {127.0.0.1:}
[--12T11::,][INFO ][o.e.n.Node               ] [4LdtI3q] started
[--12T11::,][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [4LdtI3q] Failed to clear cache for realms [[]]
[--12T11::,][INFO ][o.e.l.LicenseService     ] [4LdtI3q] license [d717e767-fe3f-4b36-bc85-cad6dbde3efc] mode [basic] - valid
[--12T11::,][INFO ][o.e.g.GatewayService     ] [4LdtI3q] recovered [] indices into cluster_state
[--12T11::,][INFO ][o.e.c.r.a.AllocationService] [4LdtI3q] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[customer][], [customer][]] ...]).

二、REST使用示例

1. 查看集群的状态
curl -X GET "localhost:9200/_cat/health?v"
epoch      timestamp cluster       status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1531365002 11:10:02  elasticsearch yellow          1         1     10  10    0    0       10             0                  -                 50.0%
注意这个status的状态是yellow，理想的状态应该是green，原因是没有配置集群。

2. 查看节点的状态
curl -X GET "localhost:9200/_cat/nodes?v"
ip        heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
127.0.0.1           20          57   3    0.07    0.06     0.06 mdi       *      4LdtI3q

3. 查看所有索引（类比mysql的库）的状态
curl -X GET 'http://localhost:9200/_cat/indices?v'
health status index    uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   bank     BO1IYzoXSpGGyYP3XKo4bA   5   1       1000            0    475.1kb        475.1kb
yellow open   customer -8lF3GAFT1WI43gR9DiYpw   5   1          5            0     17.7kb         17.7kb
有两个索引bank和customer，一个1000条数据，一个有5条数据。

4. 查看索引（customer）的状态
curl -X PUT "localhost:9200/customer?pretty"
{
  "error" : {
    "root_cause" : [
      {
        "type" : "resource_already_exists_exception",
        "reason" : "index [customer/-8lF3GAFT1WI43gR9DiYpw] already exists",
        "index_uuid" : "-8lF3GAFT1WI43gR9DiYpw",
        "index" : "customer"
      }
    ],
    "type" : "resource_already_exists_exception",
    "reason" : "index [customer/-8lF3GAFT1WI43gR9DiYpw] already exists",
    "index_uuid" : "-8lF3GAFT1WI43gR9DiYpw",
    "index" : "customer"
  },
  "status" : 400
}

5. 更新（不存在时则增加）索引的document（类比Mysql的行）
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
  "name": "John Doe"
}
'
响应如下：
{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 8, //每更新一次版本记录就+1
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 7,
  "_primary_term" : 4
}

6. 增加索引的document（和PUT增加的区别就是不用指定document的id，会随机生成一个）
curl -X POST "localhost:9200/customer/_doc?pretty" -H 'Content-Type: application/json' -d'
{
  "name": "Jane Doe"
}
'
响应如下：
{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "Bu1tjWQBZa8_pSJjP5AF", //随机生成的id
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 8,
  "_primary_term" : 4
}

7. 查看指定的document
curl -X GET "localhost:9200/customer/_doc/1?pretty"
响应结果
{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 8,
  "found" : true,
  "_source" : {
    "name" : "John Doe"
  }
}

8. 删除指定的document
curl -X DELETE "localhost:9200/customer/_doc/B-1ujWQBZa8_pSJjH5CC?pretty"
响应结果：
{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "B-1ujWQBZa8_pSJjH5CC",
  "_version" : 3,
  "result" : "not_found",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 7,
  "_primary_term" : 4
}

9. 批量插入文本数据
curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@/home/lings/accounts.json"
/home/lings/accounts.json为json文件的存放路径

10. 按照account_number升幂排序
curl -X GET "localhost:9200/bank/_search?q=*&sort=account_number:asc&pretty"
完全等效于这个命令
curl -X GET "localhost:9200/bank/_search" -H 'Content-Type: application/json' -d'
{
  "query": { "match_all": {} },
  "sort": [
    { "account_number": "asc" }
  ]
}
'

响应结果：
{
  "took" : 3, //查询花费了3ms
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1000, //一共有1000条数据
    "max_score" : null,
    "hits" : [
      {
        "_index" : "bank",
        "_type" : "_doc",
        "_id" : "0",
        "_score" : null,
        "_source" : {
          "account_number" : 0,
          "balance" : 16623,
          "firstname" : "Bradshaw",
          "lastname" : "Mckenzie",
          "age" : 29,
          "gender" : "F",
          "address" : "244 Columbus Place",
          "employer" : "Euron",
          "email" : "bradshawmckenzie@euron.com",
          "city" : "Hobucken",
          "state" : "CO"
        },
        "sort" : [
          0
        ]
      },
 ......省略中间数据
       {
        "_index" : "bank",
        "_type" : "_doc",
        "_id" : "9",
        "_score" : null,
        "_source" : {
          "account_number" : 9,
          "balance" : 24776,
          "firstname" : "Opal",
          "lastname" : "Meadows",
          "age" : 39,
          "gender" : "M",
          "address" : "963 Neptune Avenue",
          "employer" : "Cedward",
          "email" : "opalmeadows@cedward.com",
          "city" : "Olney",
          "state" : "OH"
        },
        "sort" : [
          9  //一共只返回了10个document
        ]
      }
    ]
  }
}

11. 查询住址包含Street单词的所有document
curl -X GET "localhost:9200/bank/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "query": { "match": {"address":"Street"} },
  "sort": [
    { "account_number": "desc" }
  ]
}
'
响应如下：
{
  "took" : 9,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 385, //满足条件的385个，总共还是返回10个document
    "max_score" : null,
    "hits" : [
      {
        "_index" : "bank",
        "_type" : "_doc",
        "_id" : "996",
        "_score" : null,
        "_source" : {
          "account_number" : 996,
          "balance" : 17541,
          "firstname" : "Andrews",
          "lastname" : "Herrera",
          "age" : 30,
......

12. 查询住址包含Street单词的降幂排序后的2个document
curl -X GET "localhost:9200/bank/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "query": { "match": {"address":"Street"} },
  "sort": [
    { "account_number": "desc" }
  ],
  "size":2
}
'
响应如下：
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 385,
    "max_score" : null,
    "hits" : [
      {
        "_index" : "bank",
        "_type" : "_doc",
        "_id" : "996",
        "_score" : null,
        "_source" : {
          "account_number" : 996,
          "balance" : 17541,
          "firstname" : "Andrews",
          "lastname" : "Herrera",
          "age" : 30,
          "gender" : "F",
          "address" : "570 Vandam Street",
          "employer" : "Klugger",
          "email" : "andrewsherrera@klugger.com",
          "city" : "Whitehaven",
          "state" : "MN"
        },
        "sort" : [
          996
        ]
      },
      {
        "_index" : "bank",
        "_type" : "_doc",
        "_id" : "994",
        "_score" : null,
        "_source" : {
          "account_number" : 994,
          "balance" : 33298,
          "firstname" : "Madge",
          "lastname" : "Holcomb",
          "age" : 31,
          "gender" : "M",
          "address" : "612 Hawthorne Street",
          "employer" : "Escenta",
          "email" : "madgeholcomb@escenta.com",
          "city" : "Alafaya",
          "state" : "OR"
        },
        "sort" : [
          994
        ]
      }
    ]
  }
}