Docker 部署ELK之Sentinl日志报警

前篇文章简单介绍了Docker 部署ELK，以及使用filebeat收集java日志。这篇我们介绍下日志报警配置，这里我们使用Sentinl插件。

1、修改kibana参数

进入elk容器，修改对应参数

[root@centos-mq ~]# docker exec -it elk /bin/bash
root@70f05fc990bd:/# vim /opt/kibana/config/kibana.yml
sentinl:
  settings:
    email:
      active: true
　　  #ssl: true 　　　　　　## 云服务器时打开这注释，因为云服务器会禁用25端口 
     #port：465
      user: *****@163.com　## 发件人
      password: ****　　　　## 授权码(不是密码)
      host: smtp.163.com
    report:
      active: false

2、安装Sentinl插件

Sentinl版本要选择与kibana版本一致，否则会安装失败

root@70f05fc990bd:/# /opt/kibana/bin/kibana-plugin install https://github.com/sirensolutions/sentinl/releases/download/tag-6.6.0-0/sentinl-v6.6.0.zip
Attempting to transfer from https://github.com/sirensolutions/sentinl/releases/download/tag-6.6.0-0/sentinl-v6.6.0.zip
Transferring 134770542 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Optimizing and caching browser bundles...
Plugin installation complete
root@70f05fc990bd:~# /etc/init.d/kibana restart

在docker里面下载总是失败，我从宿主机下载，然后拷贝到docker容器里：

[root@localhost duan]# pwd
/home/duan
[root@localhost duan]# docker cp /home/duan/sentinl-v6.6.0.zip elk:/opt
[root@localhost duan]# docker exec -it elk sh
# cd opt
# ls
elasticsearch  kibana  logstash  sentinl-v6.6.0.zip

安装时指定的是本地文件：

# /opt/kibana/bin/kibana-plugin install file:////opt/sentinl-v6.6.0.zip
Attempting to transfer from file:////opt/sentinl-v6.6.0.zip
Transferring 134770542 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Optimizing and caching browser bundles...
Plugin installation was unsuccessful due to error "Command failed: /opt/kibana/node/bin/node /opt/kibana/src/cli --env.name=production --optimize.useBundleCache=false --server.autoListen=false --plugins.initialize=false

FATAL CLI ERROR YAMLException: can not read an implicit mapping pair; a colon is missed at line 106, column 5:
　　 #ssl: true 　　　　　　## 云服务器时打开这注释，因 ...
^
at generateError (/opt/kibana/node_modules/js-yaml/lib/js-yaml/loader.js:160:10)
at throwError (/opt/kibana/node_modules/js-yaml/lib/js-yaml/loader.js:166:9)
at readBlockMapping (/opt/kibana/node_modules/js-yaml/lib/js-yaml/loader.js:1018:11)
at composeNode (/opt/kibana/node_modules/js-yaml/lib/js-yaml/loader.js:1315:12)
at readDocument (/opt/kibana/node_modules/js-yaml/lib/js-yaml/loader.js:1478:3)
at loadDocuments (/opt/kibana/node_modules/js-yaml/lib/js-yaml/loader.js:1538:5)
at load (/opt/kibana/node_modules/js-yaml/lib/js-yaml/loader.js:1555:19)
at Object.safeLoad (/opt/kibana/node_modules/js-yaml/lib/js-yaml/loader.js:1573:10)
at readYaml (/opt/kibana/src/core/server/config/read_config.js:25:38)
at Object.exports.getConfigFromFiles (/opt/kibana/src/core/server/config/read_config.js:50:22)
"
#

上面的配置失败了，因为kibana.yml文件的ssl配置项

#ops.interval:
sentinl:
  settings:
    email:
      active: true
      user: xxxx@.com
      password: xxxxx
      host: smtp..com
      ssl: false
    report:
      active: true
# vi kibana.yml

修改配置文件，删除sentinl插件重新安装插件：

cd /opt/kibana/bin
./kibana-plugin remove sentinl
./kibana-plugin install file:///opt/sentinl-v6.6.0.zip

安装重启完，浏览器访问kibana界面，即可看到Sentinl插件菜单

3、配置报警

Sentinl >> New >> Watcher Advanced

点击保存，会创建一个报警模板，修改模板内容如下：

{
  "actions": {
    "邮件告警": {
      "name": "日志异常",
      "throttle_period": "0h2m0s",
      "email_html": {
        "stateless": false,
        "subject": "evolut-api-gateway模块--ERROR日志",
        "priority": "medium",
        "html": "<p><i>Hi,各位同事请注意下面有 {{payload.hits.total}} 条错误信息，请查看并处理！！</i>.</p>\n<div style=\"color:grey;\">\n  <hr />\n</div>\n<div>\n<br>{{#payload.hits.hits}} <li style='color:red'><b>source:</b> {{_source.source}} </li><br><li><b>message</b>: {{_source.message}}</li><br><br>{{/payload.hits.hits}}  \n</div>",
        "to": "xiong@xxx.com",
        "from": "e@126.com"
      }
    },
    "钉钉告警模板": {
      "name": "webhook告警",
      "throttle_period": "0h2m0s",
      "webhook": {
        "priority": "medium",
        "stateless": false,
        "method": "POST",
        "host": "oapi.dingtalk.com",
        "port": "",
        "path": "/robot/send?access_token=bdf86156bcded8b10727ceff898b943ef726baaebd797f760336",
        "body": "{\r\n    \"msgtype\": \"markdown\",\r\n    \"at\": {\r\n        \"isAtAll\": \"True\"\r\n    },\r\n    \"markdown\": {\r\n        \"title\": \"异常消息\",\r\n        \"text\": \" evolut-api-gateway模块-错误日志: \\n {{#payload.hits.hits}} {{_source.message}} \r\n{{/payload.hits.hits}}\"\r\n    }\r\n}",
        "params": {
          "watcher": "{{watcher.title}}",
          "payload_count": "{{payload.hits.total}}"
        },
        "headers": {
          "Content-Type": "application/json"
        },
        "message": "生产环境异常",
        "use_https": true
      }
    }
  },
  "input": {
    "search": {
      "request": {
        "index": [
          "prd-evolut-api-gateway*"
        ],
        "body": {
          "query": {
            "bool": {
              "must": {
                "match": {
                  "message": "ERROR"
                }
              },
              "filter": {
                "range": {
                  "@timestamp": {
                    "gte": "now-5m/m",
                    "lte": "now/m",
                    "format": "epoch_millis"
                  }
                }
              }
            }
          },
          "size": ,
          "aggs": {
            "dateAgg": {
              "date_histogram": {
                "field": "@timestamp",
                "time_zone": "Asia/Shanghai",
                "interval": "1m",
                "min_doc_count":
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "script": {
      "script": "payload.hits.total >= 1"
    }
  },
  "transform": {},
  "trigger": {
    "schedule": {
      "later": "every 2 minutes"
    }
  },
  "disable": false,
  "report": false,
  "title": "evolut-api-gateway"
}

配置完成后，等待设置对应的时间，是要触发设置的报警机制，会看到报警日志发送至设定的邮箱

钉钉：

在sentinl里面加一个watcher：

Input填入：

{
"search": {
"request": {
"index": [
"*"
],
"body": {
"query": {
"bool": {
"must": [
{
"query_string": {
"analyze_wildcard": true,
"query": "\"error\""
}
},
{
"range": {
"@timestamp": {
"gte": "now-10m",
"lte": "now",
"format": "epoch_millis"
}
}
}
],
"must_not": []
}
}
}
}
}
}
condition填入：
{
"script": {
"script": "payload.hits.total > 1"
}
}
添加一个webhook的action：

以上所有配置根据自己需要修改，附上钉钉的demo地址：

https://open-doc.dingtalk.com/docs/doc.htm?spm=a219a.7629140.0.0.karFPe&treeId=257&articleId=105735&docType=1#s0

成功报警。
————————————————
版权声明：本文为CSDN博主「挑葱夫」的原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/Dragon714/article/details/80625386