elasticsearch配置集群+elk报错总结

配置ELK的时候，我平常遇到了以下几种报错情况，整理如下（持续更新中）：

1. elasticsearch启动失败

   # systemctl start elasticsearch
   Job for elasticsearch.service failed because the control process exited with error code. See "systemctl status elasticsearch.service" and "journalctl -xe" for details.
   #这个时候，直接查看系统日志，因为elasticsearch没有专门的日志审计
   tail -f /var/log/messages
   

   出现如下报错

   Dec 13 10:16:30 oldboy elasticsearch: ERROR: [1] bootstrap checks failed
   Dec 13 10:16:30 oldboy elasticsearch: [1]: initial heap size [536870912] not equal to maximum heap size [775946240]; this can cause resize pauses and prevents mlockall from locking the entire heap
   

   其实提示已经很明显了，jvm给的内存不足，那么我们直接把内存调大就可以了

   #修改jvm内存大小
   # vim /etc/elasticsearch/jvm.options
   -Xms1500m
   -Xms1500m
   #因为刚才把内存改的很小，改回来就行了
   

   如果不是使用的systemd方法启动，直接调用bin/elasticsearch 启动，那么有几点需要注意

   #1.不能使用root进行登录
   useradd elk #创建用户elk
   #2.将涉及的用户权限赋予elk
   

2. kibana显示中文乱码

   #首先查看要拉取的日志的格式是什么
   file file.txt  #在linux上查看
   以记事本打开log文件，点击另存为查看，如果显示为ANSI，那么就是gbk  #在windows上查看
   #在filebeat中配置字符集
   # vim /etc/filebeat/filebeat.yml
   filebeat.inputs:
   - type: log
     enabled: true
     paths:
       - c:\work\CA*
     encoding: gbk   #此处加入字符格式，如果是utf8，那么不需要添加
   

   继续生成测试日志，登录kibana查看，发现中文字符已经正常显示，没有乱码了。

3. es集群配置xpack启动后，创建密码失败

   [root@db01 elasticsearch]# bin/elasticsearch-setup-passwords interactive
   Failed to determine the health of the cluster running at http://10.0.0.200:9200
   Unexpected response code [503] from calling GET http://10.0.0.200:9200/_cluster/health?pretty
   Cause: master_not_discovered_exception
   It is recommended that you resolve the issues with your cluster before running elasticsearch-setup-passwords.
   It is very likely that the password changes will fail when run against an unhealthy cluster.
   Do you want to continue with the password setup process [y/N]y
   Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
   You will be prompted to enter passwords as the process progresses.
   Please confirm that you would like to continue [y/N]y
   #错误原因，因为脏数据的原因，当开始xpack的时候，集群链接失败
   #终极大招（只适用于初始创建集群，或者测试环境）
   1.停止服务
   2.删除数据目录
   3.三个节点只配置xpack.security.enabled: true,启动
   4.设置密码
   #配置文件（三台除了ip之外都一样）
   cluster.name: think
   node.name: node-1
   path.data: /var/lib/elasticsearch
   path.logs: /var/log/elasticsearch
   bootstrap.memory_lock: true
   network.host: 10.0.0.200,127.0.0.1
   http.port: 9200
   discovery.seed_hosts: ["10.0.0.200", "10.0.0.201"]
   cluster.initial_master_nodes: ["10.0.0.200", "10.0.0.201","10.0.0.202"]
   http.cors.enabled: true
   http.cors.allow-origin: "*"
   xpack.security.enabled: true
   #测试效果
   [root@db01 elasticsearch]# bin/elasticsearch-setup-passwords interactive
   Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
   You will be prompted to enter passwords as the process progresses.
   Please confirm that you would like to continue [y/N]y
   Enter password for [elastic]:
   Reenter password for [elastic]:
   Enter password for [apm_system]:
   Reenter password for [apm_system]:
   Enter password for [kibana]:
   Reenter password for [kibana]:
   Enter password for [logstash_system]:
   Reenter password for [logstash_system]:
   Enter password for [beats_system]:
   Reenter password for [beats_system]:
   Enter password for [remote_monitoring_user]:
   Reenter password for [remote_monitoring_user]:
   Changed password for user [apm_system]
   Changed password for user [kibana]
   Changed password for user [logstash_system]
   Changed password for user [beats_system]
   Changed password for user [remote_monitoring_user]
   Changed password for user [elastic]
   #成功
   

4.隔天上班又出现和标题3同样的情况，如下解决方案

#直接配上ca证书验证，开启ssl
# 设置默认的角色密码
bin/elasticsearch-setup-passwords interactive  #这一步我是不成功的，不过标题3已经创建过了，所以跳过
再elasticsearch.yml加入如下
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate # 证书验证级别
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12 # 节点证书路径
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
#创建证书
# 创建keystore文件
# bin/elasticsearch-keystore create # config文件夹下有的话这一步就不用再执行了
# 生成CA证书，一直回车
bin/elasticsearch-certutil ca (CA证书：elastic-stack-ca.p12)
# 生成节点使用的证书，一直回车
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12  (节点证书：elastic-certificates.p12)
# 创建证书保存目录，并移动到config文件下
mkdir -p /etc/elasticsearch/certs
mv elastic-certificates.p12 /etc/elasticsearch/certs
chmod 777 /etc/elasticsearch/certs   #不给授权就无法登录，可以自己测测到底给多少合适
#重启