ElasticSearch 2 (2) - Setup
ElasticSearch 2.1.1 (2) - Setup
Installation
Elasticsearch can be started using:
$ bin/elasticsearch
Running as a daemon
$ bin/elasticsearch -d
PID
The PID is written to a file called pid.
$ bin/elasticsearch -d -p pid
The kill command sends a TERM signal to the PID stored in the pid file.
$ kill `cat pid`
Another feature is the ability to pass -D or getopt long style configuration parameters directly to the script. When set, all override anything set using either JAVA_OPTS or ES_JAVA_OPTS. For example:
$ bin/elasticsearch -Des.index.refresh_interval=5s --node.name=my-node
Configuration
Environment Variables
JAVA_OPTS
ES_JAVA_OPTS > JAVA_OPTS
ES_HEAP_SIZE
ES_MIN_MEM (defaults to 256m)
ES_MAX_MEM (defaults to 1g)
It is recommended to set the min and max memory to the SAME value, and enable mlockall.
System Configuration
File Descriptors
Setting it to 32k or even 64k is recommended.
-Des.max-open-files=true
max_file_descriptors
curl localhost:9200/_nodes/stats/process?pretty
Virtual memory
Elasticsearch uses a hybrid mmapfs / niofs directory by default to store its indices. The default operating system limits on mmap counts is likely to be too low, which may result in out of memory exceptions. On Linux, you can increase the limits by running the following command as root:
sysctl -w vm.max_map_count=262144
To set this value permanently, update the vm.max_map_count setting in /etc/sysctl.conf.
Memory Settings
Most operating systems try to use as much memory as possible for file system caches and eagerly swap out unused application memory, possibly resulting in the elasticsearch process being swapped. Swapping is very bad for performance and for node stability, so it should be avoided at all costs.
Disable swap
Box -
ES_HEAP_SIZE
Linux -
Temporarily
sudo swapoff -a
Permanently (comment out swap)
/etc/fstab
Windows -
System Properties → Advanced → Performance → Advanced → Virtual memory
Configure swappiness
This reduces the kernel’s tendency to swap and should not lead to swapping under normal circumstances, while still allowing the whole system to swap in emergency conditions.
vm.swappiness
From kernel version 3.5-rc1 and above, a swappiness of 0 will cause the OOM killer to kill the process instead of allowing swapping. You will need to set swappiness to 1 to still allow swapping in emergencies.
mlockall
Linux - mlockall
bootstrap.mlockall: true
Windows - VirtualLock
Check:
curl http://localhost:9200/_nodes/process?pretty
Grant(as root):
$ ulimit -l unlimited
Directory(/tmp):
./bin/elasticsearch -Djna.tmpdir=/path/to/new/dir
mlockall might cause the JVM or shell session to exit if it tries to allocate more memory than is available!
Elasticsearch Settings
Directory
ES_HOME/config
module - elasticsearch.yml
logging - logging.yml
network
The address all network based modules will use to bind and publish to:
network :
host : 10.0.0.4
Path
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch
Cluster name
Don’t forget to give your production cluster a name, which is used to discover and auto-join other nodes:
cluster:
name: <NAME OF YOUR CLUSTER>
Don’t reuse the same cluster names in different environment
- development: logging-dev
- staging: logging-stage
- production: logging-prod
Node name
Default (Marvel Character Name)
Provided
node:
name: <NAME OF YOUR NODE>
Single Node on Machine
node:
name: ${HOSTNAME}
Configuration style
JSON
elasticsearch.json:
{
"network" : {
"host" : "10.0.0.4"
}
}
Command
$ elasticsearch -Des.network.host=10.0.0.4
Default
es.default.
Environment
{
"network" : {
"host" : "${ES_NET_HOST}"
}
}
Non-store
${prompt.text} or ${prompt.secret}
node:
name: ${prompt.text}
On execution:
Enter value for [node.name]:
Elasticsearch will not start if ${prompt.text} or ${prompt.secret} is used in the settings and the process is run as a service or in the background.
Index Settings
Index Level (YAML or JSON)
$ curl -XPUT http://localhost:9200/kimchy/ -d \
'
index:
refresh_interval: 5s
'
Node Level (elasticsearch.yml)
index :
refresh_interval: 5s
Collapsed
$ elasticsearch -Des.index.refresh_interval=5s
Logging
Log4j (log4j-extras)
Format
- .yml
- .yaml
- .json
- .properties
Deprecation
config/logging.yml
deprecation: DEBUG, deprecation_log_file
Running as a Service on Linux
Linux
ES_USER
The user to run as, defaults to elasticsearch
ES_GROUP
The group to run as, defaults to elasticsearch
ES_HEAP_SIZE
The heap size to start with
ES_HEAP_NEWSIZE
The size of the new generation heap
ES_DIRECT_SIZE
The maximum size of the direct memory
MAX_OPEN_FILES
Maximum number of open files, defaults to 65535
MAX_LOCKED_MEMORY
Maximum locked memory size. Set to "unlimited" if you use the bootstrap.mlockall option in elasticsearch.yml. You must also set ES_HEAP_SIZE.
MAX_MAP_COUNT
Maximum number of memory map areas a process may have. If you use mmapfs as index store type, make sure this is set to a high value. For more information, check the linux kernel documentation about max_map_count. This is set via sysctl before starting elasticsearch. Defaults to 65535
LOG_DIR
Log directory, defaults to /var/log/elasticsearch
DATA_DIR
Data directory, defaults to /var/lib/elasticsearch
CONF_DIR
Configuration file directory (which needs to include elasticsearch.yml and logging.yml files), defaults to /etc/elasticsearch
ES_JAVA_OPTS
Any additional java options you may want to apply. This may be useful, if you need to set the node.name property, but do not want to change the elasticsearch.yml configuration file, because it is distributed via a provisioning system like puppet or chef. Example: ES_JAVA_OPTS="-Des.node.name=search-01"
RESTART_ON_UPGRADE
Configure restart on package upgrade, defaults to false. This means you will have to restart your elasticsearch instance after installing a package manually. The reason for this is to ensure, that upgrades in a cluster do not result in a continuous shard reallocation resulting in high network traffic and reducing the response times of your cluster.
ES_GC_LOG_FILE
The absolute log file path for creating a garbage collection logfile, which is done by the JVM. Note that this logfile can grow pretty quick and thus is disabled by default.
Debian/Ubuntu
runlevels
update-rc.d
init script
/etc/init.d/elasticsearch
configuration
/etc/default/elasticsearch
after install
dpkg -i
start on boot and start up
sudo update-rc.d elasticsearch defaults 95 10
sudo /etc/init.d/elasticsearch start
Oracle JDK
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
java -version
RPM based distribution
Using chkconfig
init script
/etc/init.d/elasticsearch
configuration
/etc/sysconfig/elasticsearch
manual start
sudo /sbin/chkconfig --add elasticsearch
sudo service elasticsearch start
Using systemd
system rpm based
/etc/default/elasticsearch
configuration
/etc/sysconfig/elasticsearch
up
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
sudo /bin/systemctl start elasticsearch.service
MAX_MAP_COUNT setting
/etc/sysconfig/elasticsearch (no effect)
/usr/lib/sysctl.d/elasticsearch.conf
Running as a Service on Windows
https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-service-win.html
Directory Layout
Type(Setting) - Default Location
Description
home(path.home) - N/A
Home of elasticsearch installation.
bin(N/A) - {path.home}/bin
Binary scripts including elasticsearch to start a node.
conf(path.conf) - {path.home}/config
Configuration files including elasticsearch.yml
data(path.data) - {path.home}/data
The location of the data files of each index / shard allocated on the node. Can hold multiple locations.
logs(path.logs) - {path.home}/logs
Log files location.
plugins(path.plugins) - {path.home}/plugins
Plugin files location. Each plugin will be contained in a subdirectory.
repo(path.repo) - Not configured
Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here.
script(path.script) - {path.conf}/scripts
Location of script files.
Multiple data paths
---------------------------------
path.data: /mnt/first,/mnt/second
---------------------------------
Or array format
----------------------------------------
path.data: ["/mnt/first", "/mnt/second"]
----------------------------------------
To stripe shards across multiple disks, please use a RAID driver instead.
Default Path
deb & rpm
-----------------------------------------------------------------------------------
Type | Location Debian/Ubuntu | Location RHEL/CentOS
-----------------------------------------------------------------------------------
home | /usr/share/elasticsearch | /usr/share/elasticsearch
-----------------------------------------------------------------------------------
bin | /usr/share/elasticsearch/bin | /usr/share/elasticsearch/bin
-----------------------------------------------------------------------------------
conf | /etc/elasticsearch | /etc/elasticsearch
-----------------------------------------------------------------------------------
conf | /etc/default/elasticsearch | /etc/sysconfig/elasticsearch
-----------------------------------------------------------------------------------
data | /var/lib/elasticsearch/data | /var/lib/elasticsearch
-----------------------------------------------------------------------------------
logs | /var/log/elasticsearch | /var/log/elasticsearch
-----------------------------------------------------------------------------------
plugins | /usr/share/elasticsearch/plugins | /usr/share/elasticsearch/plugins
-----------------------------------------------------------------------------------
repo | Not configured | Not configured
-----------------------------------------------------------------------------------
script | /etc/elasticsearch/scripts | /etc/elasticsearch/scripts
-----------------------------------------------------------------------------------
zip
-----------------------------------------------------------------------------------
Type | Description | Location
-----------------------------------------------------------------------------------
home | Home of elasticsearch installation | {extract.path}
-----------------------------------------------------------------------------------
bin | scripts to start a node | {extract.path}/bin
-----------------------------------------------------------------------------------
conf | files elasticsearch.yml and logging.yml | {extract.path}/config
-----------------------------------------------------------------------------------
data | location of files of each index / shard | {extract.path}/data
-----------------------------------------------------------------------------------
logs | Log files location | {extract.path}/logs
-----------------------------------------------------------------------------------
plugins | Plugin files location. | {extract.path}/plugins
-----------------------------------------------------------------------------------
repo | Shared file system repository locations. | Not configured
-----------------------------------------------------------------------------------
script | Location of script files. | {extract.path}/config/scripts
-----------------------------------------------------------------------------------
Repositories
PGP Key
We use the PGP key D88E42B4, Elasticsearch Signing Key, with fingerprint
4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4
APT
Download and install the Public Signing Key:
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
Save the repository definition to /etc/apt/sources.list.d/elasticsearch-2.x.list:
echo "deb http://packages.elastic.co/elasticsearch/2.x/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-2.x.list
Run apt-get update and the repository is ready for use. You can install it with:
sudo apt-get update && sudo apt-get install elasticsearch
Configure Elasticsearch to automatically start during bootup. If your distribution is using SysV init, then you will need to run:
sudo update-rc.d elasticsearch defaults 95 10
Otherwise if your distribution is using systemd:
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
YUM
Download and install the public signing key:
rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
Add the following in your /etc/yum.repos.d/ directory in a file with a .repo suffix, for example elasticsearch.repo
[elasticsearch-2.x]
name=Elasticsearch repository for 2.x packages
baseurl=http://packages.elastic.co/elasticsearch/2.x/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1
And your repository is ready for use. You can install it with:
yum install elasticsearch
SysV init
chkconfig --add elasticsearch
Systemd
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
Reference
https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html
ElasticSearch 2 (2) - Setup的更多相关文章
- jar hell & elasticsearch ik 版本问题
想给es 安装一个ik 的插件, 我的es 是 2.4.0, 下载了一个版本是 1.9.5, [2016-10-09 16:56:26,248][INFO ][node ] [node-2] init ...
- Elasticsearch Configuration 中文版
##################### Elasticsearch Configuration Example ##################### # This file contains ...
- centos6.5安装elasticsearch
java下载地址:http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.htmles下载地址 : ...
- 分布式搜索引擎Elasticsearch的简单使用
官方网址:https://www.elastic.co/products/elasticsearch/ 一.特性 1.支持中文分词 2.支持多种数据源的全文检索引擎 3.分布式 4.基于lucene的 ...
- Elasticsearch配置文件说明
一.Cluster setting Cluster indices.ttl.interval 允许设置多久过期的文件会被自动删除.默认值是60秒. indices.cache.filter.siz ...
- ElasticSearch学习-centos下安装
1.安装java运行环境(jre) //这里我安装了jdk 其实只需要安装jre就可以了 0)cd /usr;mkdir /usr/java; cd java 1)wget http://downlo ...
- elasticsearch单机多实例环境部署
elasticsearch的功能,主要用在搜索领域,这里,我来研究这个,也是项目需要,为公司开发了一款CMS系统,网站上的搜索栏功能,我打算采用elasticsearch来实现. elasticsea ...
- ELK——安装 logstash 2.2.0、elasticsearch 2.2.0 和 Kibana 3.0
本文内容 Elasticsearch logstash Kibana 参考资料 本文介绍安装 logstash 2.2.0 和 elasticsearch 2.2.0,操作系统环境版本是 CentOS ...
- ELK初学搭建(elasticsearch)
ELK初学搭建(elasticsearch) elasticsearch logstash kibana ELK初学搭建 elasticsearch 1.环境准备 centos6.8_64 mini ...
随机推荐
- Deprecated: Methods with the same name as their class will not be constructors in a future version of PHP
<?php class Car { var $color = "add"; function Car($color="green") { $this-&g ...
- 解决IDEA因分配内存而引起的卡顿
解决IDEA分配内存不足引起卡顿的问题 在使用IDEA的过程中,经常会运行一段时间后程序卡顿.一段代码可能要敲很久或者出现死机状态,严重影响代码书写速度.经查阅资料,IDEA的自动分配内存最大只有75 ...
- flask 蓝本(blueprint)
蓝本(blueprint) 一.基本概念: 将视图方法模块化,既当大量的视图函数放在一个文件中,很明显是不合适,最好的方案是根据功能将路由合理的划分到不同的文件中:而蓝本就是为了解决这个问题而出现的. ...
- 蓝桥杯历届试题 危险系数(dfs或者并查集求无向图关于两点的割点个数)
Description 抗日战争时期,冀中平原的地道战曾发挥重要作用. 地道的多个站点间有通道连接,形成了庞大的网络.但也有隐患,当敌人发现了某个站点后,其它站点间可能因此会失去联系. 我们来定义一个 ...
- 五,ESP8266 TCP服务器多连接(基于Lua脚本语言)
https://www.cnblogs.com/yangfengwu/p/7524326.html 一些时间去准备朋友的元器件了... 接着写,,争取今天写完所有的文章,,因为答应了朋友下周5之前要做 ...
- python高速排序
import random def rand(n): for i in range(n): yield random.randint(0,1000) #创建一个随机数列表 def createList ...
- kettle学习笔记(七)——kettle流程步骤与应用步骤
一.概述 流程主要用来控制数据流程与数据流向 应用则是提供一些工具类 二.流程步骤 1.ETL元数据注入 类似Java中的反射,在设计时不知道文件名.文件位置等,在真正执行时才知道具体的一些配置等信息 ...
- 20155216 Exp4 恶意代码分析
20155216 Exp4 恶意代码分析 实践内容 使用schtasks指令监控系统运行 先在C盘目录下建立一个netstatlog.bat文件和netstatlog.txt文件,将记录的联网结果格式 ...
- # 20155337《网络对抗》Exp6 信息搜集与漏洞扫描
20155337<网络对抗>Exp6 信息搜集与漏洞扫描 实践目标 (1)各种搜索技巧的应用 (2)DNS IP注册信息的查询 (3)基本的扫描技术:主机发现.端口扫描.OS及服务版本探测 ...
- Luogu P1558 色板游戏
(此题与POJ2777重题) 为了加深对线段树的记忆,然后开始搞这道题. TM的WA了一下午就是发现x可能大于y(然而题目里说的还很清楚,我TM没看见) 这道题只需要在线段树的板子上改一些地方就可以了 ...