filebeat+kafka失败

filebeat端配置

#----------------------------- Kafka output --------------------------------
output.kafka:
  # initial brokers for reading cluster metadata
  hosts: ["10.88.128.88:9092"]
  version: "0.8.2"

  topic: 'abc'
  #partition.round_robin:
  #  reachable_only: false

  required_acks:
  compression: none
  #max_message_bytes: 

报错如下：

// ::24.060269 log.go:: WARN producer/broker/ state change to [closing] because dial tcp: lookup test43 on 172.16.2.1:: no such host
// ::24.060361 log.go:: WARN producer/leader/rokid/ state change to [retrying-]
// ::24.060405 log.go:: WARN producer/leader/rokid/ abandoning broker
// ::24.060450 log.go:: WARN producer/broker/ shut down
x2017// ::24.160582 log.go:: WARN client/metadata fetching metadata for [rokid] from broker 10.88.128.88:
// ::24.164350 log.go:: WARN producer/broker/ starting up
// ::24.164388 log.go:: WARN producer/broker/ state change to [open] on rokid/
// ::24.164405 log.go:: WARN producer/leader/rokid/ selected broker
// ::24.164505 log.go:: WARN producer/leader/rokid/ state change to [flushing-]
// ::24.164516 log.go:: WARN producer/leader/rokid/ state change to [normal]
i2017// ::24.185439 log.go:: WARN Failed to connect to broker test43:: dial tcp: lookup test43 on 172.16.2.1:: no such host
// ::24.185488 log.go:: WARN producer/broker/ state change to [closing] because dial tcp: lookup test43 on 172.16.2.1:: no such host
// ::24.185534 log.go:: WARN producer/leader/rokid/ state change to [retrying-]
// ::24.185552 log.go:: WARN producer/leader/rokid/ abandoning broker
// ::24.185592 log.go:: WARN producer/broker/ shut down
// ::24.285740 log.go:: WARN client/metadata fetching metadata for [rokid] from broker 10.88.128.88:
// ::24.287170 log.go:: WARN producer/broker/ starting up
// ::24.287191 log.go:: WARN producer/broker/ state change to [open] on rokid/
// ::24.287205 log.go:: WARN producer/leader/rokid/ selected broker
// ::24.287300 log.go:: WARN producer/leader/rokid/ state change to [flushing-]
// ::24.287311 log.go:: WARN producer/leader/rokid/ state change to [normal]
// ::24.304521 log.go:: WARN Failed to connect to broker test43:: dial tcp: lookup test43 on 172.16.2.1:: no such host

上网查，filebeat解析的时候要解析到主机名，

所以绑定hosts如下

　　10.88.128.88 test43

问题搞定

按照报错，filebeat应该是先以配置的IP获取hostname，然后按照hostname连接kafka，如果通过hostname找不到对方服务器，则报错，所以需要在filebeat机器上绑定对方机器的hosts，暂不理解filebeat为什么要这么做。

如果目标机器中hosts文件书写有误，也会报错（filebeat向kafka中传递数据的时候还会检查目标机器的hosts文件）

 

开始还以为kafka版本太新，修改为filebeat目前支持的稳定版0.8.2都出问题，反复查，绑定hosts问题搞定。

参考网址：http://www.iyunv.com/thread-350295-1-1.html