Java操作Kafka执行不成功
使用kafka-clients
操作kafka
始终不成功,原因不清楚,下面贴出相关代码及配置,请懂得指点一下,谢谢!
环境及依赖
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.10.2.0</version>
</dependency>
JDK
版本为1.8、Kafka
版本为2.12-0.10.2.0
,服务器使用CentOS-7
构建。
测试代码
TestBase.java
public class TestBase {
protected Logger log = LoggerFactory.getLogger(this.getClass());
protected String kafka_server = "192.168.60.160:9092" ;
protected String topic = "zlikun_topic";
}
ProducerTest.java
public class ProducerTest extends TestBase {
protected Properties props = new Properties();
@Before
public void init() {
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafka_server);
props.put(ProducerConfig.ACKS_CONFIG, "all");
props.put(ProducerConfig.RETRIES_CONFIG, 0);
props.put(ProducerConfig.BATCH_SIZE_CONFIG, 16384);
props.put(ProducerConfig.LINGER_MS_CONFIG, 1);
props.put(ProducerConfig.BUFFER_MEMORY_CONFIG, 33554432);
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.PARTITIONER_CLASS_CONFIG ,MyPartitioner.class) ;
}
@Test
public void test() throws InterruptedException {
KafkaProducer<String, String> producer = new KafkaProducer<>(props);
// 发送消息
for (int i = 0; i < 10; i++) {
producer.send(new ProducerRecord<String, String>(topic, Integer.toString(i), Integer.toString(i)), new Callback() {
@Override
public void onCompletion(RecordMetadata recordMetadata, Exception e) {
if (e == null) {
System.out.printf("offset = %d ,partition = %d \n", recordMetadata.offset() ,recordMetadata.partition());
} else {
log.error("send error !" ,e);
}
}
});
}
TimeUnit.SECONDS.sleep(3);
producer.close();
}
}
ConsumerTest.java
public class ConsumerTest extends TestBase {
private Properties props = new Properties();
@Before
public void init() {
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, kafka_server);
props.put(ConsumerConfig.GROUP_ID_CONFIG ,"zlikun") ;
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "true");
props.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "1000");
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
}
@Test
public void test() {
Consumer<String, String> consumer = new KafkaConsumer<>(props);
consumer.subscribe(Arrays.asList(topic));
// consumer.assign(Arrays.asList(new TopicPartition(topic, 1)));
while (true) {
ConsumerRecords<String, String> records = consumer.poll(100);
for (ConsumerRecord<String, String> record : records) {
System.out.printf("offset = %d, key = %s, value = %s%n", record.offset(), record.key(), record.value());
}
}
}
}
问题
# 测试topic为手动创建
$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 4 --topic zlikun_topic
控制台输出信息
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s) for zlikun_topic-3: 30042 ms has passed since batch creation plus linger time
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s) for zlikun_topic-3: 30042 ms has passed since batch creation plus linger time
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s) for zlikun_topic-2: 30042 ms has passed since batch creation plus linger time
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s) for zlikun_topic-2: 30042 ms has passed since batch creation plus linger time
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 3 record(s) for zlikun_topic-1: 30043 ms has passed since batch creation plus linger time
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 3 record(s) for zlikun_topic-1: 30043 ms has passed since batch creation plus linger time
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 3 record(s) for zlikun_topic-1: 30043 ms has passed since batch creation plus linger time
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 3 record(s) for zlikun_topic-0: 30046 ms has passed since batch creation plus linger time
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 3 record(s) for zlikun_topic-0: 30046 ms has passed since batch creation plus linger time
[kafka-producer-network-thread | producer-1] ERROR com.zlikun.mq.ProducerTest - send error !
org.apache.kafka.common.errors.TimeoutException: Expiring 3 record(s) for zlikun_topic-0: 30046 ms has passed since batch creation plus linger time
Java操作Kafka执行不成功 >> java
这个答案描述的挺清楚的:
http://www.goodpm.net/postreply/java/1010000008863969/Java操作Kafka执行不成功.html
Java操作Kafka执行不成功的更多相关文章
- Java操作Kafka执行不成功的解决方法,Kafka Broker Advertised.Listeners属性的设置
创建Spring Boot项目继承Kafka,向Kafka发送消息始终不成功.具体项目配置如下: <?xml version="1.0" encoding="UTF ...
- Java操作Kafka
java操作kafka非常的简单,然后kafka也提供了很多缺省值,一般情况下我们不需要修改太多的参数就能使用.下面我贴出代码. pom.xml <dependency> <grou ...
- 在java.ext.dirs中使用环境变量导致crontab执行不成功的问题及解决
在java.ext.dirs中使用环境变量导致crontab执行不成功的问题及解决 Table of Contents 1. java.ext.dirs的使用和环境变量 2. 问题:在crontab中 ...
- kafka实战教程(python操作kafka),kafka配置文件详解
kafka实战教程(python操作kafka),kafka配置文件详解 应用往Kafka写数据的原因有很多:用户行为分析.日志存储.异步通信等.多样化的使用场景带来了多样化的需求:消息是否能丢失?是 ...
- 【MongoDB for Java】Java操作MongoDB
上一篇文章: http://www.cnblogs.com/hoojo/archive/2011/06/01/2066426.html介绍到了在MongoDB的控制台完成MongoDB的数据操作,通过 ...
- Java操作Oracle
public class DBCon { // 数据库驱动对象 public static final String DRIVER = "oracle.jdbc.driver.OracleD ...
- JAVA操作ORACLE数据库的存储过程
一.任务提出 JAVA操作oracle11g存储过程实验需要完成以下几个实例: 1.调用没有返回参数的过程(插入记录.更新记录) 2.有返回参数的过程 3.返回列表的过程 4.返回带分页的列表的过程. ...
- JAVA操作MongoDB数据库
1. 首先,下载MongoDB对Java支持的驱动包 驱动包下载地址:https://github.com/mongodb/mongo-java-driver/downloads 2.Java操作Mo ...
- Java JDBC下执行SQL的不同方式、参数化预编译防御
相关学习资料 http://zh.wikipedia.org/wiki/Java数据库连接 http://lavasoft.blog.51cto.com/62575/20588 http://blog ...
随机推荐
- 页面footer在底部
页脚动态贴在底部需要满足以下两个条件: 当主体的内容高度不超过可视区域高度的时候,页脚贴在页面底部. 当主体的内容高度超过可视区域高度的时候,页脚将按正常布局. 方法一:footer高度固定+绝对定位 ...
- JS 封装一个求n~m的求和函数
var a = 0; cc(2,10); function cc(n,m){ for(var i =n;i<(m+1);i++){ a = a + ...
- nyoj303-序号交换
序号互换 时间限制:1000 ms | 内存限制:65535 KB 难度:2 描述 Dr.Kong设计了一个聪明的机器人卡多,卡多会对电子表格中的单元格坐标快速计算出来.单元格的行坐标是由数字编号 ...
- matplotlib 显示两张图片,折线图 和 scipy
显示两张图片的代码: import numpy as np from scipy.misc import imread, imsave, imresize import matplotlib.pypl ...
- java深入的单例模式
在GoF的23种设计模式中,单例模式是比较简单的一种.然而,有时候越是简单的东西越容易出现问题.下面就单例设计模式详细的探讨一下. 所谓单例模式,简单来说,就是在整个应用中保证只有一个类的实例存在 ...
- 【hdu 4135】Co-prime
[题目链接]:http://acm.hdu.edu.cn/showproblem.php?pid=4135 [题意] 让你求出[a..b]这个区间内和N互质的数的个数; [题解] 利用前缀和,求出[1 ...
- Cookie 工具类
一.导入 jar 包 <dependency> <groupId>javax.servlet</groupId> <artifactId>servlet ...
- ElasticSearch[v6.2] 在实际项目中的应用
摘要:本文所讲述的内容,为ElasticSearch(以下简称ES)全文搜索引擎在实际大数据项目的应用:ES的底层是开源库 Lucene.但是,你没法直接用 Lucene,必须自己写代码去调用它的接口 ...
- 2015 Multi-University Training Contest 10 hdu 5411 CRB and Puzzle
CRB and Puzzle Time Limit: 2000/1000 MS (Java/Others) Memory Limit: 65536/65536 K (Java/Others)To ...
- 如何用Vim提高开发效率
即可 ●输入m获取到文章目录 推荐↓↓↓ C/C++编程 更多推荐<18个技术类公众微信> 涵盖:程序人生.算法与数据结构.黑客技术与网络安全.大数据技术.前端开发.Java.Python ...