centos7.6下的python3.6.9虚拟环境安装elastalert
centos7.6安装python3.6.9+elastalert .编译安装python3..9环境
# 安装依赖
yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel
# 获取编译安装python3.6.9
mkdir -p /usr/local/python3
wget https://www.python.org/ftp/python/3.6.9/Python-3.6.9.tgz
tar xf Python-3.6..tgz
cd Python-3.6.
./configure --prefix=/usr/local/python3
make && make install
ln -s /usr/local/python3/bin/python-3.6.9/bin/python3.6 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3 .安装virtualenv虚拟环境
pip3 install virtualenv
# 创建存放虚拟环境的目录
mkdir -p /usr/local/venv_py3.6_elastalert-0.2. # 创建纯净的虚拟环境
cd /usr/local
git clone https://github.com/Yelp/elastalert.git
cd /usr/local/elastalert /usr/local/python-3.6./bin/virtualenv --no-site-packages --python=/usr/local/python-3.6./bin/python3. /usr/local/venv_py3.6_elastalert-0.2.
[root@eus-kibana-elastalert-:/usr/local/venv_py3.6_elastalert-0.2.]# source bin/activate
(venv_py3.6_elastalert-0.2.) [root@eus-kibana-elastalert-:/usr/local/venv_py3.6_elastalert-0.2.]# .在虚拟的python3.6环境中安装alasticalert
# 指定库,安装依赖,否则可能安装失败 (venv_py3.6_elastalert-0.2.) [root@eus-kibana-elastalert-:/usr/local/elastalert]# pip install -r requirements.txt -i https://pypi.python.org/simple # 安装主程序,否则无法使用 elastalert-create-index 命令
(venv_py3.6_elastalert-0.2.) [root@eus-kibana-elastalert-:/usr/local/elastalert]# python setup.py install # 运行 elastalert-create-index 配置
(venv_py3.6_elastalert-0.2.) [root@eus-kibana-elastalert-:/usr/local/elastalert]# elastalert-create-index
Enter Elasticsearch host: 172.30.0.62
Enter Elasticsearch port:
Use SSL? t/f: f
Enter optional basic-auth username (or leave blank):
Enter optional basic-auth password (or leave blank):
Enter optional Elasticsearch URL prefix (prepends a string to the URL of every request):
New index name? (Default elastalert_status)
New alias name? (Default elastalert_alerts)
Name of existing index to copy? (Default None)
Elastic Version: 7.3.
Reading Elastic index mappings:
Reading index mapping 'es_mappings/6/silence.json'
Reading index mapping 'es_mappings/6/elastalert_status.json'
Reading index mapping 'es_mappings/6/elastalert.json'
Reading index mapping 'es_mappings/6/past_elastalert.json'
Reading index mapping 'es_mappings/6/elastalert_error.json'
New index elastalert_status created
Done!
(venv_py3.6_elastalert-0.2.) [root@eus-kibana-elastalert-:/usr/local/elastalert]# # 报错
(venv_py3.6_elastalert-0.2.) [root@eus-kibana-elastalert-:/usr/local/elastalert]# elastalert-test-rule example_rules/my_rule.yml File "/usr/local/venv_py3.6_elastalert-0.2.1/lib/python3.6/site-packages/tzlocal/unix.py", line , in _get_localzone
utils.assert_tz_offset(tz)
File "/usr/local/venv_py3.6_elastalert-0.2.1/lib/python3.6/site-packages/tzlocal/utils.py", line , in assert_tz_offset
raise ValueError(msg)
ValueError: Timezone offset does not match system offset: != -. Please, check your config files. # 代码和系统时区不匹配,重新设置为上海时区
(venv_py3.6_elastalert-0.2.) [root@eus-kibana-elastalert-:/usr/local/elastalert]# timedatectl set-timezone Asia/Shanghai
(venv_py3.6_elastalert-0.2.) [root@eus-kibana-elastalert-:/usr/local/elastalert]# elastalert-test-rule example_rules/my_rule.yml
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
To send them but remain verbose, use --verbose instead.
WARNING:elasticsearch:GET http://172.30.0.62:19200/logstash-*/_search?ignore_unavailable=true&size=1 [status:400 request:0.004s]
Error running your filter:
RequestError(, 'parsing_exception', {'error': {'root_cause': [{'type': 'parsing_exception', 'reason': '[term] query malformed, no start_object after query name', 'line': , 'col': }], 'type': 'parsing_exception', 'reason': '[term] query malformed, no start_object after query name', 'line': , 'col': }, 'status': })
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
To send them but remain verbose, use --verbose instead.
rules loaded
INFO:apscheduler.scheduler:Adding job tentatively -- it will be properly scheduled when the scheduler starts
WARNING:elasticsearch:GET http://172.30.0.62:19200/logstash-*/_search?_source_includes=%2A%2C%40timestamp&ignore_unavailable=true&scroll=30s&size=10000 [status:400 request:0.003s]
ERROR:root:Error running query: RequestError(, 'parsing_exception', '[term] query malformed, no start_object after query name') Would have written the following documents to writeback index (default is elastalert_status): elastalert_error - {'message': "Error running query: RequestError(400, 'parsing_exception', '[term] query malformed, no start_object after query name')", 'traceback': ['Traceback (most recent call last):', ' File "/usr/local/venv_py3.6_elastalert-0.2.1/lib/python3.6/site-packages/elastalert-0.2.1-py3.6.egg/elastalert/elastalert.py", line 384, in get_hits', ' **extra_args', ' File "/usr/local/venv_py3.6_elastalert-0.2.1/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped', ' return func(*args, params=params, **kwargs)', ' File "/usr/local/venv_py3.6_elastalert-0.2.1/lib/python3.6/site-packages/elasticsearch/client/__init__.py", line 819, in search', ' "GET", _make_path(index, "_search"), params=params, body=body', ' File "/usr/local/venv_py3.6_elastalert-0.2.1/lib/python3.6/site-packages/elasticsearch/transport.py", line 350, in perform_request', ' timeout=timeout,', ' File "/usr/local/venv_py3.6_elastalert-0.2.1/lib/python3.6/site-packages/elasticsearch/connection/http_requests.py", line 156, in perform_request', ' self._raise_error(response.status_code, raw_data)', ' File "/usr/local/venv_py3.6_elastalert-0.2.1/lib/python3.6/site-packages/elasticsearch/connection/base.py", line 181, in _raise_error', ' status_code, error_message, additional_info', "elasticsearch.exceptions.RequestError: RequestError(400, 'parsing_exception', '[term] query malformed, no start_object after query name')"], 'data': {'rule': 'eus-log-elasticsearch-cluster-alert', 'query': {'query': {'bool': {'filter': {'bool': {'must': [{'range': {'@timestamp': {'gt': '2019-09-17T05:06:25.831477Z', 'lte': '2019-09-17T05:21:25.831477Z'}}}, {'term': None}, {'query_string': {'query': 'message: error'}}]}}}}, 'sort': [{'@timestamp': {'order': 'asc'}}]}}} .配置elastalert
##############全局配置
[root:/usr/local/elastalert#cp config.yaml.example config.yaml
#存放elastalert 规则的文件夹,你的elastalert 放到哪里就放到哪里就行了
rules_folder: /usr/local/elastalert/example_rules #Elastalert 多久去查询一下根据定义的规则去elasticsearch 查询是否有符合规则的字段,如果有就会触发报警,如果没有就等待下一次时间再检查,时间定义的单位从周到秒都可以,具体定义方法如下。
run_every:
#seconds:
minutes:
#hours:
#days:
#weeks: #当查询开始一直到结束,最大的缓存时间。
buffer_time:
minutes: #你的Elasticsearch ip地址
es_host: 172.30.0.52 #Elasticsearch 的端口
es_port: #这个是elastalert 在es里边写的index
# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: elastalert_status #如果alert当时没有发出去重试多久之后放弃发送;
alert_time_limit:
days: [root@eus-kibana-elastalert-:/usr/local/elastalert]# egrep -v '^#|^$' config.yaml
rules_folder: example_rules
run_every:
minutes:
buffer_time:
minutes:
es_host: 172.30.0.62
es_port:
writeback_index: elastalert_status
writeback_alias: elastalert_alerts
alert_time_limit:
days: ##############rules 的定义
[root@ws-elk-cluster01:/usr/local/elastalert]#cp example_frequency.yaml my_rule.yaml
vi my_rule.yaml
# Alert when the rate of events exceeds a threshold
#Elasticsearch 机器
es_host: 192.168.115.65 #Elasticsearch 端口
es_port: #如果elasticsearch 有认证,填写用户名和密码的地方
# (Optional) basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword #rule name 必须是独一的,不然会报错,这个定义完成之后,会成为报警邮件的标题
# (Required)
# Rule name, must be unique
name: ws-elk-cluster-alert #配置一种数据验证的方式,有 any,blacklist,whitelist,change,frequency,spike,flatline,new_term,cardinality
any:只要有匹配就报警;
blacklist:compare_key字段的内容匹配上 blacklist数组里任意内容;
whitelist:compare_key字段的内容一个都没能匹配上whitelist数组里内容;
change:在相同query_key条件下,compare_key字段的内容,在 timeframe范围内 发送变化;
frequency:在相同 query_key条件下,timeframe 范围内有num_events个被过滤出 来的异常;
spike:在相同query_key条件下,前后两个timeframe范围内数据量相差比例超过spike_height。其中可以通过spike_type设置具体涨跌方向是- up,down,both 。还可以通过threshold_ref设置要求上一个周期数据量的下限,threshold_cur设置要求当前周期数据量的下限,如果数据量不到下限,也不触发;
flatline:timeframe 范围内,数据量小于threshold 阈值;
new_term:fields字段新出现之前terms_window_size(默认30天)范围内最多的terms_size (默认50)个结果以外的数据;
cardinality:在相同 query_key条件下,timeframe范围内cardinality_field的值超过 max_cardinality 或者低于min_cardinality # (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
#我配置的是frequency,这个需要两个条件满足,在相同 query_key条件下,timeframe 范围内有num_events个被过滤出来的异常
type: frequency #这个index 是指再kibana 里边的index,支持正则匹配,支持多个index,同时如果嫌麻烦直接* 也可以。
index: customer*
#index: es-nginx*,winlogbeat* #时间出发的次数
num_events: #和上边的参数关联,也就是说在4分钟内出发5次会报警
timeframe:
minutes: #这个还是非常关键的地方,就是你希望程序的message里边出现了什么样的关键字就报警,这个其实就是elasticsearch 的query语句,支持 AND&OR等。
filter:
- query:
query_string:
query: "message: 错误 OR Error" #一但需要报警用那种方式报警,支持如下的方式,同时官方支持自定义,我用常规的邮件方式作为报警方式。
alert:
- "email"
#在邮件正文会显示你定义的alert_text
alert_text: "Ref Log http://192.168.254.194"
#报警邮箱的smtp server
smtp_host: mail.chinasoft.cn
#报警邮箱的smtp 端口
smtp_port:
#需要把认证信息写到额外配置文件里,需要user和password两个属性
smtp_auth_file: /usr/local/elastalert/example_rules/smtp_auth_file.yaml
email_reply_to:jack@.com
from_addr: jack@.com #接受报警邮箱的地址,可以写多个,当然后边搞个邮件组最好了。
# (required, email specific)
# a list of email addresses to send alerts to
email:
- "jack@163.com" [root@eus-kibana-elastalert-:/usr/local/elastalert/example_rules]# egrep -v '^#|^$' my_rule.yml
es_host: 172.30.0.62
es_port:
name: eus-log-elasticsearch-cluster-alert
type: frequency
index: filebeats-log*
num_events:
timeframe:
hours:
filter:
- term:
- query:
query_string:
query: "message: error"
alert:
- "email"
email:
- "jack@chinasoft.cn"
alert_text: "Ref Log http://172.30.0.62"
smtp_host: mail.chinasoft.cn
smtp_port:
smtp_auth_file: /usr/local/elastalert/example_rules/smtp_auth_file.yaml
email_reply_to: jack@chinasoft.cn
from_addr: jack@chinasoft.cn ######################smtp认证文件
[root@ws-elk-cluster01:/usr/local/elastalert]#vi smtp_auth_file.yaml
user: "jack"
password: "jack123" #通过elastalert-test-rule 测试一下我们写的rule 是否有问题
[root@ws-elk-cluster01:/usr/local/elastalert/example_rules]# elastalert-test-rule my_rule.yaml #配置检查成功之后,我们就可以把程序跑起来了,把所有的日志直接打在前端,这样方便验证
/usr/local/venv_py3.6_elastalert-0.2./bin/python3. -m elastalert.elastalert --verbose --rule /usr/local/elastalert/example_rules/my_rule.yaml
centos7.6下的python3.6.9虚拟环境安装elastalert的更多相关文章
- windows和linux下 Python2,Python3 的环境及安装
目录 windows和linux下 Python2,Python3 的环境及安装 window下安装 一. 手动安装 二. pip安装 linux下 安装 更新Python 笔者有话 windows和 ...
- python笔记:学习设置Python虚拟环境+配置 virtualenvwarpper+创建Python3.6的虚拟环境+安装numpy
虚拟环境它是一个虚拟化,从电脑独立开辟出来的环境.就是借助虚拟机docker来把一部分内容独立出来,我们把这部分独立出来的东西称作“容器”,在这个容器中,我们可以只安装我们需要的依赖包,各个容器之间互 ...
- 全网最全的Windows下Python2 / Python3里正确下载安装用来向微信好友发送消息的itchat库(图文详解)
不多说,直接上干货! 建议,你用Anaconda2或Anaconda3. 见 全网最全的Windows下Anaconda2 / Anaconda3里正确下载安装用来向微信好友发送消息的itchat库( ...
- centos7.x下环境搭建(一)--yum方式安装mysql5.7
前两天因为数据库被黑客攻击,导致数据被删除,数据库被损坏,系统重新安装了一下,所以环境也需要重新再搭一遍,包括mysql.nodejs.git.nginx和redis的安装.由于之前安装的mysql安 ...
- centos7.4下的python3.6的安装
1.系统环境 :centos 7.4 最小化安装 2.安装过程 yum install wget 安装下载工具 wget https://www.python.org/ftp/python/ ...
- centos7.2下部署 python3
安装Python3 1.环境准备 yum -y install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel read ...
- Win10下创建Python3.7创建虚拟环境以及安装Flask框架
鉴于现在看到的很多虚拟环境创建以及flask框架安装方式需要通过dos命令来做,虽然比较常用,但是每次运行都要激活虚拟环境,相对比较麻烦,而现在利用pycharm大可不必如此. 1.安装破解版pych ...
- centos7环境下apache2.2.34的编译安装
.获取apache2..34的源码包 http://archive.apache.org/dist/httpd/httpd-2.2.34.tar.gz .获取apache的编译参数 apache的编译 ...
- ubuntu16.04下使用python3开发时,安装pip3与scrapy,升级pip3
1)安装pip3: sudo apt-get install python3-pip 2)安装scrapy sudo pip3 install scrapy 若出现版本过低问题: pip3 insta ...
随机推荐
- Mac系统安装JDK
MAC 安装 JDK: 这篇文章主要为在MAC苹果系统下安装JDK1.8并配置系统环境变量. 主要分为以下步骤: 到Oracle官网下载JDK1.8安装包. 打开获取到的安装包按步骤安装到系统上. 配 ...
- golang读写文件
1. 标准输入输出 os提供了标准输入输出文件: Stdin = NewFile(uintptr(syscall.Stdin), "/dev/stdin") Stdout = Ne ...
- 异常错误:在可以调用 OLE 之前,必须将当前线程设置为单线程单元(STA)模式
最近做一个蛋疼的东西就是C#调用windows API 来操作一个摄像头,自动处理一些东西.要用到剪切板复制 粘贴功能,即 Clipboard.SetDataObject(filedic, true) ...
- 性能:Transform层面
数据处理的并行度 1.BlockRDD的分区数 (1)通过Receiver接受数据的特点决定 (2)也可以自己通过repartition设置 2.ShuffleRDD的分区数 (1)默认的分区数为sp ...
- 对于模块加载:ES6、CommonJS、AMD、CMD的区别
运行和编译的概念 编译包括编译和链接两步. 编译,把源代码翻译成机器能识别的代码或者某个中间状态的语言. 比如java只有JVM识别的字节码,C#中只有CLR能识别的MSIL.还简单的作一些比如检查有 ...
- How to Start Up an Open Source Company
https://evolveum.com/start-open-source-company/ Evolveum is a successful open source company now. We ...
- c#——ref 和 out 的区别
一个用关键字 ref 标示,一个用 out 标示. 牵扯到数据是引用类型还是值类型. 一般用这两个关键字你是想调用一个函数将某个值类型的数据通过一个函数后进行更改.传 out 定义的参数进去的时候这个 ...
- *arg和**kwarg作用
*args:可以理解为只有一列的表格,长度不固定. **kwargs:可以理解为字典,长度也不固定. 1.函数调用里的*arg和**kwarg: (1) *arg:元组或列表 ...
- 阿里巴巴编程规约--digest
所谓卫语句,如果某个条件极其罕见,就应该单独检查该条件,并在该条件为真时立刻从函数中返回.这样的单独检查常常被称为“卫语句”. 微服务之间将DTO,Req放到一个单独的项目中,相关的项目都依赖这个底层 ...
- JavaScript基础03——函数的作用域及变量提升
1.作用域 作用域,变量在函数内部作用的范围/区域.有函数的地方就有作用域. 2.局部作用域和全局作用域 function fn(){ var a = 1; } console.log(a); / ...