Elasticsearch之settings和mappings的意义

Elasticsearch之settings和mappings（图文详解）

 

Elasticsearch之settings和mappings的意义

　　简单的说，就是

　　settings是修改分片和副本数的。

　　mappings是修改字段和类型的。

　　记住，可以用url方式来操作它们，也可以用java方式来操作它们。建议用url方式，因为简单很多。

1、ES中的settings

　　查询索引库的settings信息

[hadoop@HadoopMaster elasticsearch-2.4.3]$ curl -XGET http://192.168.80.10:9200/zhouls/_settings?pretty
{
"zhouls" : {
"settings" : {
"index" : {
"creation_date" : "1488203759467",
"uuid" : "Sppm-db_Qm-OHptOC7vznw",
"number_of_replicas" : "1",
"number_of_shards" : "5",
"version" : {
"created" : "2040399"
}
}
}
}
}
[hadoop@HadoopMaster elasticsearch-2.4.3]$

settings修改索引库默认配置

　　例如：分片数量，副本数量

　　查看：curl -XGET http://192.168.80.10:9200/zhouls/_settings?pretty

　　操作不存在索引：curl -XPUT '192.168.80.10:9200/liuch/' -d'{"settings":{"number_of_shards":3,"number_of_replicas":0}}'

　　操作已存在索引：curl -XPUT '192.168.80.10:9200/zhouls/_settings' -d'{"index":{"number_of_replicas":1}}'

总结：就是，不存在索引时,可以指定副本和分片，如果已经存在,则只能修改副本。

　　在创建新的索引库时，可以指定索引分片的副本数。默认是1，这个很简单

2、ES中的mappings

　　ES的mapping如何用？什么时候需要手动，什么时候需要自动？

Mapping,就是对索引库中索引的字段名称及其数据类型进行定义，类似于mysql中的表结构信息。不过es的mapping比数据库灵活很多，它可以动态识别字段。一般不需要指定mapping都可以，因为es会自动根据数据格式识别它的类型，如果你需要对某些字段添加特殊属性（如：定义使用其它分词器、是否分词、是否存储等），就必须手动添加mapping。

　　我们在es中添加索引数据时不需要指定数据类型，es中有自动影射机制，字符串映射为string，数字映射为long。通过mappings可以指定数据类型是否存储等属性。

　　查询索引库的mapping信息

[hadoop@HadoopMaster elasticsearch-2.4.3]$ curl -XGET http://192.168.80.10:9200/zhouls/emp/_mapping?pretty
{
"zhouls" : {
"mappings" : {
"emp" : {
"properties" : {
"name" : {
"type" : "string"
},
"score" : {
"type" : "long"
},
"type" : {
"type" : "string"
}
}
}
}
}
}
[hadoop@HadoopMaster elasticsearch-2.4.3]$

mappings修改字段相关属性

　　例如：字段类型，使用哪种分词工具啊等，如下：

注意：下面可以使用indexAnalyzer定义分词器，也可以使用index_analyzer定义分词器

操作不存在的索引
　　curl -XPUT '192.168.80.10:9200/zhouls' -d'{"mappings":{"emp":{"properties":{"name":{"type":"string","analyzer": "ik_max_word"}}}}}'
操作已存在的索引
　　curl -XPOST http://192.168.80.10:9200/zhouls/emp/_mapping -d'{"properties":{"name":{"type":"string","analyzer": "ik_max_word"}}}'

　　也许我上面这样写，很多人不太懂，我下面，就举个例子。（大家必须要会）

 

第一步：先编辑tvcount.json文件

　　内容如下（做了笔记）：

{
    "settings":{　　　　#settings是修改分片和副本数的
           "number_of_shards":3,　　　　#分片为3
           "number_of_replicas":0　　　　#副本数为0
        },
    "mappings":{　　　　#mappings是修改字段和类型的
          "tvcount":{
               "dynamic":"strict",
                   "_all":{"enabled":false},
                   "properties":{
                   "tvname":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
　　　　　　　　　　　　　　　　　　如，string类型，analyzed索引，ik_max_word分词器
                   "director":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
           "actor":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
           "allnumber":{"type":"string","index":"not_analyzed"},
           "tvtype":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
           "description":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
           "pic":{"type":"string","index":"not_analyzed"}
                   }
          }
        }
 }

　　即，tvname（电视名称）　　　　director（导演）   　　　　actor（主演）　　　　　　allnumber（总播放量）

　　　　tvtype（电视类别）　　　　description（描述）

‘’

[hadoop@master elasticsearch-2.4.0]$ ll
total 52
drwxrwxr-x 2 hadoop hadoop  4096 Jul  6 20:25 bin
drwxrwxr-x 3 hadoop hadoop  4096 Jul  6 20:27 config
drwxrwxr-x 2 hadoop hadoop  4096 Apr 21 14:19 lib
-rw-rw-r-- 1 hadoop hadoop 11358 Aug 24  2016 LICENSE.txt
drwxrwxr-x 5 hadoop hadoop  4096 Aug 29  2016 modules
-rw-rw-r-- 1 hadoop hadoop   150 Aug 24  2016 NOTICE.txt
drwxrwxr-x 6 hadoop hadoop  4096 Jul  6 15:33 plugins
-rw-rw-r-- 1 hadoop hadoop  8700 Aug 24  2016 README.textile
-rw-rw-r-- 1 hadoop hadoop   195 Jul  1 12:18 requests
[hadoop@master elasticsearch-2.4.0]$ vim tvcount.json

{
    "settings":{
           "number_of_shards":3,
           "number_of_replicas":0
        },
    "mappings":{
          "tvcount":{
               "dynamic":"strict",
                   "_all":{"enabled":false},
                   "properties":{
                   "tvname":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
                   "director":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
           "actor":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
           "allnumber":{"type":"string","index":"not_analyzed"},
           "tvtype":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
           "description":{"type":"string","index":"analyzed","analyzer":"ik_max_word","search_analyzer": "ik_max_word"},
           "pic":{"type":"string","index":"not_analyzed"}
                   }
          }
        }
}

http://192.168.80.145:9200/_plugin/head/

　第二步：创建mapping

　　这里，因为，之前，我们是在/home/hadoop/app/elasticsearch-2.4.0下，这个目录下有我们刚之前写的tvcount.json，所以可以直接

curl -XPOST 'http://master:9200/tv' -d @tvcount.json

　　不然的话，就需要用绝对路径

[hadoop@master elasticsearch-2.4.0]$ pwd
/home/hadoop/app/elasticsearch-2.4.0
[hadoop@master elasticsearch-2.4.0]$ ll
total 56
drwxrwxr-x 2 hadoop hadoop  4096 Jul  6 20:25 bin
drwxrwxr-x 3 hadoop hadoop  4096 Jul  6 20:27 config
drwxrwxr-x 2 hadoop hadoop  4096 Apr 21 14:19 lib
-rw-rw-r-- 1 hadoop hadoop 11358 Aug 24  2016 LICENSE.txt
drwxrwxr-x 5 hadoop hadoop  4096 Aug 29  2016 modules
-rw-rw-r-- 1 hadoop hadoop   150 Aug 24  2016 NOTICE.txt
drwxrwxr-x 6 hadoop hadoop  4096 Jul  6 15:33 plugins
-rw-rw-r-- 1 hadoop hadoop  8700 Aug 24  2016 README.textile
-rw-rw-r-- 1 hadoop hadoop   195 Jul  1 12:18 requests
-rw-rw-r-- 1 hadoop hadoop  1022 Jul  6 22:27 tvcount.json
[hadoop@master elasticsearch-2.4.0]$ curl -XPOST 'http://master:9200/tv' -d @tvcount.json
{"acknowledged":true}[hadoop@master elasticsearch-2.4.0]$
[hadoop@master elasticsearch-2.4.0]$
[hadoop@master elasticsearch-2.4.0]$ 

　　简单的说，就是

　　settings是修改分片和副本数的。

　　mappings是修改字段和类型的。

　　具体，见我的博客

Elasticsearch之settings和mappings（图文详解）

　　然后，再来查询下

[hadoop@master elasticsearch-2.4.0]$ pwd
/home/hadoop/app/elasticsearch-2.4.0
[hadoop@master elasticsearch-2.4.0]$ curl -XGET http://master:9200/tv/_settings?pretty
{
  "tv" : {
    "settings" : {
      "index" : {
        "creation_date" : "1499351407949",
        "uuid" : "O30Uk9uRTlGLRVfbO26gUQ",
        "number_of_replicas" : "0",
        "number_of_shards" : "3",
        "version" : {
          "created" : "2040099"
        }
      }
    }
  }
}
[hadoop@master elasticsearch-2.4.0]$ 

　　然后，再来查看mapping（mappings是修改字段和类型的）

[hadoop@master elasticsearch-2.4.0]$ pwd
/home/hadoop/app/elasticsearch-2.4.0
[hadoop@master elasticsearch-2.4.0]$ curl -XGET http://master:9200/tv/_mapping?pretty
{
  "tv" : {
    "mappings" : {
      "tvcount" : {
        "dynamic" : "strict",
        "_all" : {
          "enabled" : false
        },
        "properties" : {
          "actor" : {
            "type" : "string",
            "analyzer" : "ik_max_word"
          },
          "allnumber" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "description" : {
            "type" : "string",
            "analyzer" : "ik_max_word"
          },
          "director" : {
            "type" : "string",
            "analyzer" : "ik_max_word"
          },
          "pic" : {
            "type" : "string",
            "index" : "not_analyzed"
          },
          "tvname" : {
            "type" : "string",
            "analyzer" : "ik_max_word"
          },
          "tvtype" : {
            "type" : "string",
            "analyzer" : "ik_max_word"
          }
        }
      }
    }
  }
}
[hadoop@master elasticsearch-2.4.0]$ 

　　

　　说简单点就是，tvcount.json里已经初步设置好了settings和mappings。

　　然后启动hdfs、启动hbase

　　　　这里，很简单，不多说。

[hadoop@master elasticsearch-2.4.0]$ cd $HADOOP_HOME
[hadoop@master hadoop-2.6.0]$ jps
6261 Jps
2451 QuorumPeerMain
4893 Elasticsearch
[hadoop@master hadoop-2.6.0]$ sbin/start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
17/07/06 23:02:59 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [master]
master: starting namenode, logging to /home/hadoop/app/hadoop-2.6.0/logs/hadoop-hadoop-namenode-master.out
slave2: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0/logs/hadoop-hadoop-datanode-slave2.out
slave1: starting datanode, logging to /home/hadoop/app/hadoop-2.6.0/logs/hadoop-hadoop-datanode-slave1.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /home/hadoop/app/hadoop-2.6.0/logs/hadoop-hadoop-secondarynamenode-master.out
17/07/06 23:06:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
starting resourcemanager, logging to /home/hadoop/app/hadoop-2.6.0/logs/yarn-hadoop-resourcemanager-master.out
slave2: starting nodemanager, logging to /home/hadoop/app/hadoop-2.6.0/logs/yarn-hadoop-nodemanager-slave2.out
slave1: starting nodemanager, logging to /home/hadoop/app/hadoop-2.6.0/logs/yarn-hadoop-nodemanager-slave1.out
[hadoop@master hadoop-2.6.0]$ jps
6721 ResourceManager
6987 Jps
6390 NameNode
2451 QuorumPeerMain
6579 SecondaryNameNode
4893 Elasticsearch
[hadoop@master hadoop-2.6.0]$ 

[hadoop@slave1 elasticsearch-2.4.0]$ jps
5725 Elasticsearch
2296 QuorumPeerMain
6615 NodeManager
6529 DataNode
6683 Jps
[hadoop@slave1 elasticsearch-2.4.0]$ 

[hadoop@slave2 elasticsearch-2.4.0]$ jps
6826 NodeManager
5407 Elasticsearch
7015 Jps
6748 DataNode
2229 QuorumPeerMain
[hadoop@slave2 elasticsearch-2.4.0]$ 

[hadoop@master hadoop-2.6.0]$ cd $HBASE_HOME
[hadoop@master hbase]$ bin/start-hbase.sh
starting master, logging to /home/hadoop/app/hbase/logs/hbase-hadoop-master-master.out
slave2: regionserver running as process 7175. Stop it first.
slave1: starting regionserver, logging to /home/hadoop/app/hbase/bin/../logs/hbase-hadoop-regionserver-slave1.out
[hadoop@master hbase]$ jps
6721 ResourceManager
7214 HMaster
6390 NameNode
2451 QuorumPeerMain
6579 SecondaryNameNode
4893 Elasticsearch
7327 Jps
[hadoop@master hbase]$ 

[hadoop@slave1 hbase]$ jps
7210 Jps
7145 HRegionServer
5725 Elasticsearch
2296 QuorumPeerMain
6615 NodeManager
6529 DataNode
6969 HMaster
[hadoop@slave1 hbase]$ 

[hadoop@slave2 hbase]$ jps
6826 NodeManager
5407 Elasticsearch
7470 Jps
7337 HMaster
6748 DataNode
7175 HRegionServer
2229 QuorumPeerMain
[hadoop@slave2 hbase]$ 

　　打开进入hbase shell

[hadoop@master hbase]$ bin/hbase shell
2017-07-06 23:41:03,412 INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.98.19-hadoop2, r1e527e73bc539a04ba0fa4ed3c0a82c7e9dd7d15, Fri Apr 22 19:07:24 PDT 2016

hbase(main):001:0> 

　　查询一下有哪些库

hbase(main):001:0> list
TABLE
2017-07-06 23:51:21,204 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/app/hbase-0.98.19/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/app/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
0 row(s) in 266.1210 seconds

=> []

　　

　　如果tvcount数据库已经存在的话可以删除掉

hbase(main):002:0> disable 'tvcount'

ERROR: Table tvcount does not exist.

Here is some help for this command:
Start disable of named table:
  hbase> disable 't1'
  hbase> disable 'ns1:t1'

hbase(main):003:0> drop 'tvcount'

ERROR: Table tvcount does not exist.

Here is some help for this command:
Drop the named table. Table must first be disabled:
  hbase> drop 't1'
  hbase> drop 'ns1:t1'

hbase(main):004:0> list
TABLE
0 row(s) in 1.3770 seconds

=> []
hbase(main):005:0> 

　　然后，启动mysql数据库，创建数据库创建表

　　进一步，可以见

http://www.cnblogs.com/zlslch/p/6746922.html