Go语言之高级篇beego框架之日志收集系统
一、日志收集系统架构设计
图1
图2
二、开发环境
1、安装jdk
jdk-8u51-windows-x64.exe
安装目录：C:\Program Files\jdk8
2、安装zookeeper
zookeeper-3.3.6.tar.gz
安装目录： C:\Program Files\zookeeper-3.3.6
配置：
C:\Program Files\zookeeper-3.3.6\conf
把配置文件：zoo_sample.cfg，改名成：zoo.cfg
修改部署路径为你安装的路径。
配置环境变量：
C:\Program Files\zookeeper-3.3.6\bin
#启动服务：
进行目录，直接启动服务：C:\Program Files\zookeeper-3.3.6\bin
直接运行服务名称: zkServer.cmd
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:39,153 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53568
2019-02-20 10:52:39,154 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53568
2019-02-20 10:52:39,155 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53568
2019-02-20 10:52:39,158 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:39,161 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:39,163 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53568 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:41,214 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53570
2019-02-20 10:52:41,216 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53570
2019-02-20 10:52:41,217 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53570
2019-02-20 10:52:41,220 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:41,224 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:41,226 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53570 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:42,438 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53571
2019-02-20 10:52:42,439 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53571
2019-02-20 10:52:42,441 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53571
2019-02-20 10:52:42,444 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:42,448 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:42,449 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53571 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:44,219 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53572
2019-02-20 10:52:44,220 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53572
2019-02-20 10:52:44,221 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53572
2019-02-20 10:52:44,223 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:44,227 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:44,229 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53572 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:45,462 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53573
2019-02-20 10:52:45,464 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53573
2019-02-20 10:52:45,465 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53573
2019-02-20 10:52:45,468 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:45,473 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:45,475 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53573 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:46,769 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53574
2019-02-20 10:52:46,770 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53574
2019-02-20 10:52:46,772 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53574
2019-02-20 10:52:46,775 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:46,779 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:46,781 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53574 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:48,284 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53575
2019-02-20 10:52:48,285 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53575
2019-02-20 10:52:48,286 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53575
2019-02-20 10:52:48,289 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:48,293 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:48,295 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53575 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:50,069 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53576
2019-02-20 10:52:50,071 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53576
2019-02-20 10:52:50,072 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53576
2019-02-20 10:52:50,075 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:50,080 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:50,081 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53576 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:51,547 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53577
2019-02-20 10:52:51,548 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53577
2019-02-20 10:52:51,550 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53577
2019-02-20 10:52:51,552 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:51,557 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:51,559 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53577 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:53,315 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53579
2019-02-20 10:52:53,316 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53579
2019-02-20 10:52:53,318 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53579
2019-02-20 10:52:53,321 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:53,325 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@639] - Exception causing close of session 0x16908b8a4920000 due to java
.io.IOException: 远程主机强迫关闭了一个现有的连接。
2019-02-20 10:52:53,326 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53579 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:54,537 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53580
2019-02-20 10:52:54,539 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53580
2019-02-20 10:52:54,540 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53580
2019-02-20 10:52:54,545 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:54,551 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] - EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:52:54,552 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53580 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:56,153 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53581
2019-02-20 10:52:56,154 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53581
2019-02-20 10:52:56,156 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53581
2019-02-20 10:52:56,159 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:56,163 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] - EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:52:56,164 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53581 which had
sessionid 0x16908b8a4920000
2019-02-20 10:52:58,023 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53582
2019-02-20 10:52:58,024 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53582
2019-02-20 10:52:58,026 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53582
2019-02-20 10:52:58,028 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:58,032 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] - EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:52:58,034 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53582 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:52:59,424 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53584
2019-02-20 10:52:59,425 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53584
2019-02-20 10:52:59,425 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53584
2019-02-20 10:52:59,428 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:52:59,431 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] - EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:52:59,431 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53584 which had
sessionid 0x16908b8a4920000
2019-02-20 10:53:01,246 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53585
2019-02-20 10:53:01,247 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53585
2019-02-20 10:53:01,249 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53585
2019-02-20 10:53:01,252 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:53:01,259 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] - EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:53:01,260 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53585 which had
sessionid 0x16908b8a4920000
2019-02-20 10:53:02,677 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53587
2019-02-20 10:53:02,678 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53587
2019-02-20 10:53:02,679 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53587
2019-02-20 10:53:02,682 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:53:02,685 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] - EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:53:02,686 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53587 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:53:04,384 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53588
2019-02-20 10:53:04,385 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53588
2019-02-20 10:53:04,387 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53588
2019-02-20 10:53:04,390 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:53:04,399 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] - EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:53:04,400 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /127.0.0.1:53588 which had
sessionid 0x16908b8a4920000
2019-02-20 10:53:06,078 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /0:0:0:0:0:0:0:1:53589
2019-02-20 10:53:06,079 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /0:0:0:0
:0:0:0:1:53589
2019-02-20 10:53:06,081 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /0:0:0:0:0:0:0:1:53589
2019-02-20 10:53:06,084 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
2019-02-20 10:53:06,089 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@634] - EndOfStreamException: Unable to read additional data from client
sessionid 0x16908b8a4920000, likely client has closed socket
2019-02-20 10:53:06,090 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1435] - Closed socket connection for client /0:0:0:0:0:0:0:1:53589 whic
h had sessionid 0x16908b8a4920000
2019-02-20 10:53:07,834 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn$Factory@251] - Accepted socket connection from /127.0.0.1:53590
2019-02-20 10:53:07,835 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@770] - Client attempting to renew session 0x16908b8a4920000 at /127.0.0
.1:53590
2019-02-20 10:53:07,836 - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOSe
rverCnxn@1580] - Established session 0x16908b8a4920000 with negotiated timeout 6
000 for client /127.0.0.1:53590
2019-02-20 10:53:07,838 - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKe
eperServer@593] - Dropping packet at server of type 14
3、kafka
kafka_2.12-2.1.1.tgz
安装目录：D:\run_file\kafka_2.12-2.1.1 （不能这在C:\Program Files\ 下面启动服务会报错。）
#修改配置文件
D:\run_file\kafka_2.12-2.1.1\config
修改为本机IP地址：
advertised.host.name=172.28.5.25
#配置日志地址：
########### Log Basics ############# # A comma separated list of directories under which to store log files
#log.dirs=/tmp/kafka-logs
log.dirs=D:/run_file/kafka_2.12-2.1.1/kafka-logs #增加这行代码
启动服务：
进入安装目录中：
在命令行下：
cd D:\run_file\kafka_2.12-2.1.1 ./bin/windows/kafka-server-start.bat ./config/server.preperties
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSock
NIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1145
[2019-02-20 10:53:06,087] INFO [ZooKeeperClient] Waiting until connected. (kaf
.zookeeper.ZooKeeperClient)
[2019-02-20 10:53:06,089] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.Z
KeeperClient)
[2019-02-20 10:53:06,189] INFO [ZooKeeperClient] Waiting until connected. (kaf
.zookeeper.ZooKeeperClient)
[2019-02-20 10:53:06,190] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.Z
KeeperClient)
[2019-02-20 10:53:06,190] INFO [ZooKeeperClient] Waiting until connected. (kaf
.zookeeper.ZooKeeperClient)
[2019-02-20 10:53:06,191] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.Z
KeeperClient)
[2019-02-20 10:53:07,832] INFO Opening socket connection to server localhost/1
.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.
ache.zookeeper.ClientCnxn)
[2019-02-20 10:53:07,834] INFO Socket connection established to localhost/127.
0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2019-02-20 10:53:07,836] WARN Connected to an old server; r-o mode will be un
ailable (org.apache.zookeeper.ClientCnxnSocket)
[2019-02-20 10:53:07,837] INFO Session establishment complete on server localh
t/127.0.0.1:2181, sessionid = 0x16908b8a4920000, negotiated timeout = 6000 (or
apache.zookeeper.ClientCnxn)
[2019-02-20 10:53:07,844] INFO [SocketServer brokerId=0] Started processors fo
1 acceptors (kafka.network.SocketServer)
[2019-02-20 10:53:07,853] INFO Kafka version : 2.1.1 (org.apache.kafka.common.
ils.AppInfoParser)
[2019-02-20 10:53:07,853] INFO Kafka commitId : 21234bee31165527 (org.apache.k
ka.common.utils.AppInfoParser)
[2019-02-20 10:53:07,856] INFO [KafkaServer id=0] started (kafka.server.KafkaS
ver)
[2019-02-20 10:53:12,993] INFO [GroupMetadataManager brokerId=0] Removed 0 exp
ed offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
二、tail
在cmd中安装：
go get github.com/hpcloud/tail
#代码：
创建一个my.log的文件
路径：
D:\GoFiles\my.log
示例：
vi my.log
############################# Producer Basics ############################# # list of brokers used for bootstrapping knowledge about the rest of the cluster
# format: host1:port1,host2:port2 ...
bootstrap.servers=localhost:9092 # specify the compression codec for all data generated: none, gzip, snappy, lz4, zstd
compression.type=none # name of the partitioner class for partitioning events; default partition spreads data randomly
#partitioner.class= # the maximum amount of time the client will wait for the response of a request
#request.timeout.ms= # how long `KafkaProducer.send` and `KafkaProducer.partitionsFor` will block for
#max.block.ms= # the producer will wait for up to the given delay to allow other records to be sent so that the sends can be batched together
#linger.ms= # the maximum size of a request in bytes
#max.request.size= # the default batch size in bytes when batching multiple records sent to a partition
#batch.size= # the total bytes of memory the producer can use to buffer records waiting to be sent to the server
#buffer.memory=
示例：把文件中内容读出来
vi main.go
package main import (
"fmt"
"time" "github.com/hpcloud/tail"
) func main() {
filename := "D:\\GoFiles\\my.log"
tails, err := tail.TailFile(filename, tail.Config{
ReOpen: true,
Follow: true,
//Location: &tail.SeekInfo{Offset: 0, Whence: 2},
MustExist: false,
Poll: true,
}) if err != nil {
fmt.Println("tail file err:", err)
return
} var msg *tail.Line
var ok bool
for true {
msg, ok = <-tails.Lines
if !ok {
fmt.Printf("tail file close reopen,filename:%s\n", tails, filename)
time.Sleep(100 * time.Millisecond)
continue
}
fmt.Println("msg:", msg.Text)
}
}
执行结果：
#编译
#在终端中输入：
D:\GoFiles\src\log_collect\tail>go build D:\GoFiles\src\log_collect\tail #运行
D:\GoFiles\src\log_collect\tail>tail.exe #返回结果：
############################# Producer Basics ############################# msg:
msg: # list of brokers used for bootstrapping knowledge about the rest of the cl
uster
msg: # format: host1:port1,host2:port2 ...
msg: bootstrap.servers=localhost:9092
msg:
msg: # specify the compression codec for all data generated: none, gzip, snappy,
lz4, zstd
msg: compression.type=none
msg:
msg: # name of the partitioner class for partitioning events; default partition
spreads data randomly
msg: #partitioner.class=
msg:
msg: # the maximum amount of time the client will wait for the response of a req
uest
msg: #request.timeout.ms=
msg:
msg: # how long `KafkaProducer.send` and `KafkaProducer.partitionsFor` will bloc
k for
msg: #max.block.ms=
msg:
msg: # the producer will wait for up to the given delay to allow other records t
o be sent so that the sends can be batched together
msg: #linger.ms=
msg:
msg: # the maximum size of a request in bytes
msg: #max.request.size=
msg:
msg: # the default batch size in bytes when batching multiple records sent to a
partition
msg: #batch.size=
msg:
msg: # the total bytes of memory the producer can use to buffer records waiting
to be sent to the server
msg: #buffer.memory=
#
二、els安装
1、安装Elasticsearch
elasticsearch-5.5.0.msi
安装目录
C:\Program Files\Elastic\Elasticsearch
#启动目录
C:\Program Files\Elastic\Elasticsearch\bin
运行：
elasticsearch.exe
#访问： http://127.0.0.1:9200/
2、kafka（windows版本安装）
软件部署：
kibana-5.5.1-windows-x86.zip
解压到下面目录中：
C:\Program Files (x86)\kibana
#修改配置文件
elasticsearch.url: "http://127.0.0.1:9200" #指向es的启动页面
#启动kibana
C:\Program Files (x86)\kibana\bin
运行：
kibana.bat
示例：
运行程序的前提条件：
先安装elastic.v2
安装包：
go get gopkg.in/olivere/elastic.v2
使用方法：
import "gopkg.in/olivere/elastic.v2"
目录：
代码：
vi main.go
package main import (
"fmt" elastic "gopkg.in/olivere/elastic.v2"
) type Tweet struct {
User string
Message string
} func main() {
client, err := elastic.NewClient(elastic.SetSniff(false), elastic.SetURL("http://127.0.0.1:9200/"))
if err != nil {
fmt.Println("connect es error", err)
return
} fmt.Println("conn es succ")
for i := 0; i < 10000; i++ {
tweet := Tweet{User: "olivere", Message: "Take Five"}
_, err = client.Index().
Index("twitter").
Type("tweet").
Id(fmt.Sprintf("%d", i)).
BodyJson(tweet).
Do()
if err != nil {
// Handle error
panic(err)
return
}
} fmt.Println("insert success")
}
#打开页面：
http://localhost:5601
输入： twitter --->再点Discover就可以看到下面页面。
3、
Go语言之高级篇beego框架之日志收集系统的更多相关文章
- Go语言之高级篇beego框架安装与使用
一.beego框架 1.beego框架简介 beego 是一个快速开发 Go 应用的 HTTP 框架,他可以用来快速开发 API.Web 及后端服务等各种应用,是一个 RESTful 的框架,主要设计 ...
- Go语言之高级篇beego框架之layui框架应用
1.layui前端框架 参考地址:https://www.layui.com
- Go语言之高级篇beego框架之配置beego环境
1.配置beego环境 进入部署目录中 软件部署目录: 把要部署的软件代码,放在src目录下面. 启动项目
- Go语言之高级篇Beego框架之爬虫项目实战
一.爬虫项目 1.爬虫基础 a.网页上面会有相同的数据 b.去重处理 布隆过滤器哈希存储 c.标签匹配: 正则表达式beautiful soup或lxml这种标签提取库 d.动态内容 phantomj ...
- Go语言之高级篇beego框架之config、httplib、context
一.httplib 1.配置文件解析 这是一个用来解析文件的库,它的设计思路来自于 database/sql,目前支持解析的文件格式有 ini.json.xml.yaml,可以通过如下方式进行安装: ...
- Go语言之高级篇beego框架之view
1.基本语法 go统一使用了{{ 和 }}作为左右标签,没有其它的标签符号. 如果你想要修改为其它符号,可以修改配置文件. 使用.来访问当前位置的上下文 使用$来引用当前模板根级的上下文 2.使用方法 ...
- Go语言之高级篇beego框架之controller调用model
一.controller调用model 开发规范,就该把对数据库的操作写在model文件夹中. 示例: views/main.go package main import ( _ "web/ ...
- Go语言之高级篇beego框架之model设计构造查询
一.model设计构造查询 QueryBuilder 提供了一个简便,流畅的 SQL 查询构造器.在不影响代码可读性的前提下用来快速的建立 SQL 语句. QueryBuilder 在功能上与 ORM ...
- Go语言之高级篇beego框架之模型（Models）
一.模型(Models) 1.beego-orm的相关特性 支持 Go 的所有类型存储 -轻松上手,采用简单的 CRUD 风格 -自动 Join 关联表 跨数据库兼容查询 允许直接使用 SQL 查询/ ...
随机推荐
- Centos7X部署Zabbix监控
一:yum安装LAMP环境 zabbix-server端防火墙配置(可以选择iptables -F清空) iptables -A INPUT -m state --state NEW -m tcp - ...
- JavaScript将数字转换为大写金额
用JavaScript将数字转换为大写金额,好了 0.0 To code! var digitUppercase = function(n) { var fraction = ['角', '分']; ...
- BZOJ5291/洛谷P4458/LOJ#2512 [Bjoi2018]链上二次求和 线段树
原文链接http://www.cnblogs.com/zhouzhendong/p/9031130.html 题目传送门 - LOJ#2512 题目传送门 - 洛谷P4458 题目传送门 - BZOJ ...
- springmvc+ajax——第三讲（post请求）
在ajax01.html中增加个input标签: 在ajax的js中增加: 在controller中仍然使用getParamter():
- mysql order by多个字段
Mysql order by 多字段排序 mysql单个字段降序排序: select * from table order by id desc; mysql单个字段升序排序: select * fr ...
- 12306登录爬虫 cookies版本
import requests import re import base64 cookies = None # 进入主页,保留cookies login_url = 'https://kyfw.12 ...
- 浏览器 User-Agent 大全
一.基础知识 Http Header之User-Agent User Agent中文名为用户代理,是Http协议中的一部分,属于头域的组成部分,User Agent也简称UA.它是一个特殊字符串头,是 ...
- TF之RNN：matplotlib动态演示之基于顺序的RNN回归案例实现高效学习逐步逼近余弦曲线—Jason niu
import tensorflow as tf import numpy as np import matplotlib.pyplot as plt BATCH_START = 0 TIME_STEP ...
- ContentProvider工作原理
--摘自<android插件化开发指南> 1.系统管理类app,比如手机助手,有机会频繁使用ContentProvider 2.通讯录或者短信数据,是以ContentProvider的形式 ...
- 一、网络编程-UDP传输协议及socket套接字使用
知识点基本介绍:1.网络通信协议一般就是UDP和TCP俩种传输协议,这一章先说UDP,UDP是一种比较简单的传输协议,如qq使用的就是UDP 2.ip:ip就是标记网络中中的一台电脑 ...