As we all know that Kafka is very fast, much faster than most of its competitors. So what’s the reason here? Avoid Random Disk Access Kafka writes everything onto the disk in order and consumers fetch data in order too. So disk access always works se…

At Walmart.com in the U.S. and at Walmart's 11 other websites around the world, we provide seamless shopping experience where products are sold by: Own Merchants for Walmart.com & Walmart Stores Suppliers for Online & Stores Sellers on Walmart's m…

关于我们为什么需要Schema Registry? 参考, https://www.confluent.io/blog/how-i-learned-to-stop-worrying-and-love-the-schema-part-1/ https://www.confluent.io/blog/schema-registry-kafka-stream-processing-yes-virginia-you-really-need-one/ https://www.confluent.io/bl…

Apache Kafka is an attractive service because it's conceptually simple and powerful. It's easy to understand writing messages to a log in one place, then reading messages from that log in another place. This simplicity not only allows for a nice sepa…

This article is an in-depth tutorial for using Kafka to move data from PostgreSQL to Hadoop HDFS via JDBC connections. Read this eGuide to discover the fundamental differences between iPaaS and dPaaS and how the innovative approach of dPaaS gets to t…

This is a common question asked by many Kafka users. The goal of this post is to explain a few important determining factors and provide a few simple formulas. More Partitions Lead to Higher Throughput The first thing to understand is that a topic pa…

producer接口: /** * Licensed to the Apache Software Foundation (ASF) under one or more * contributor license agreements. See the NOTICE file distributed with * this work for additional information regarding copyright ownership. * The ASF licenses this…

Flume-ng Flume是一个分布式.可靠.和高可用的海量日志采集.聚合和传输的系统. Flume的文档可以看http://flume.apache.org/FlumeUserGuide.html 官方的英文文档 介绍的比较全面. 不过这里写写自己的见解 这个是flume的架构图 从上图可以看到几个名词: Agent: 一个Agent包含Source.Channel.Sink和其他的组件.Flume就是一个或多个Agent构成的. Source:数据源.简单的说就是agent获取数据的入口…

Introduction Apache Kafka is a distributed publish-subscribe messaging system. It was originally developed at LinkedIn Corporation and later on became a part of Apache project. Kafka is a fast, scalable, distributed in nature by its design, partition…

[This article was originally written by Yves Trudeau.] http://java.dzone.com/articles/exploring-message-brokers Message brokers are not regularly covered here but are, nonetheless, important web-related technologies. Some time ago, I was asked by one…