https://kafka.apache.org/intro.html

Kafka as a Messaging System

How does Kafka's notion of streams compare to a traditional enterprise messaging system?

【队列有扩展性,不支持多订阅者 --- 发布者-订阅者 反之   queue publish-subscribe 】

【 scale processing  multi-subscriber】

【queues aren't multi-subscriber—once one process reads the data it's gone】

【.Publish-subscribe allows you broadcast data to multiple processes, but has no way of scaling processing since every message goes to every subscriber】

Messaging traditionally has two models: queuing and publish-subscribe. In a queue, a pool of consumers may read from a server and each record goes to one of them; in publish-subscribe the record is broadcast to all consumers. Each of these two models has a strength and a weakness. The strength of queuing is that it allows you to divide up the processing of data over multiple consumer instances, which lets you scale your processing. Unfortunately, queues aren't multi-subscriber—once one process reads the data it's gone. Publish-subscribe allows you broadcast data to multiple processes, but has no way of scaling processing since every message goes to every subscriber.

【 consumer group  解决了上述2个问题】

【群内扩展,群间广播】

【不同消费群是可以订阅同一个主题的】

The consumer group concept in Kafka generalizes these two concepts. As with a queue the consumer group allows you to divide up processing over a collection of processes (the members of the consumer group). As with publish-subscribe, Kafka allows you to broadcast messages to multiple consumer groups.

【Kafka's model is that every topic can scale processing and is also multi-subscriber】

The advantage of Kafka's model is that every topic has both these properties—it can scale processing and is also multi-subscriber—there is no need to choose one or the other.

Kafka has stronger ordering guarantees than a traditional messaging system, too.

A traditional queue retains records in-order on the server, and if multiple consumers consume from the queue then the server hands out records in the order they are stored. However, although the server hands out records in order, the records are delivered asynchronously to consumers, so they may arrive out of order on different consumers. This effectively means the ordering of the records is lost in the presence of parallel consumption. Messaging systems often work around this by having a notion of "exclusive consumer" that allows only one process to consume from a queue, but of course this means that there is no parallelism in processing.

【(t,p,o)   t:p>=o 】

【By having a notion of parallelism—the partition—within the topics, Kafka is able to provide both ordering guarantees and load balancing over a pool of consumer processes. 】

【partitions 】

【一话题,多分区:一个分区同时只被一个消费者消费,保证顺序;多分区,保证平行】

Kafka does it better. By having a notion of parallelism—the partition—within the topics, Kafka is able to provide both ordering guarantees and load balancing over a pool of consumer processes. This is achieved by assigning the partitions in the topic to the consumers in the consumer group so that each partition is consumed by exactly one consumer in the group. By doing this we ensure that the consumer is the only reader of that partition and consumes the data in order. Since there are many partitions this still balances the load over many consumer instances. Note however that there cannot be more consumer instances in a consumer group than partitions.

(t,p,o) t:p>=o there cannot be more consumer instances in a consumer group than partitions的更多相关文章

  1. In-Memory:内存数据库

    在逝去的2016后半年,由于项目需要支持数据的快速更新和多用户的高并发负载,我试水SQL Server 2016的In-Memory OLTP,创建内存数据库实现项目的负载需求,现在项目接近尾声,系统 ...

  2. Tomcat一个BUG造成CLOSE_WAIT

    之前应该提过,我们线上架构整体重新架设了,应用层面使用的是Spring Boot,前段日子因为一些第三方的原因,略有些匆忙的提前开始线上的内测了.然后运维发现了个问题,服务器的HTTPS端口有大量的C ...

  3. Oracle分析函数入门

    一.Oracle分析函数入门 分析函数是什么?分析函数是Oracle专门用于解决复杂报表统计需求的功能强大的函数,它可以在数据中进行分组然后计算基于组的某种统计值,并且每一组的每一行都可以返回一个统计 ...

  4. Hangfire项目实践分享

    Hangfire项目实践分享 目录 Hangfire项目实践分享 目录 什么是Hangfire Hangfire基础 基于队列的任务处理(Fire-and-forget jobs) 延迟任务执行(De ...

  5. Sql Server系列:分区表操作

    1. 分区表简介 分区表在逻辑上是一个表,而物理上是多个表.从用户角度来看,分区表和普通表是一样的.使用分区表的主要目的是为改善大型表以及具有多个访问模式的表的可伸缩性和可管理性. 分区表是把数据按设 ...

  6. SQL Server表分区

    什么是表分区 一般情况下,我们建立数据库表时,表数据都存放在一个文件里. 但是如果是分区表的话,表数据就会按照你指定的规则分放到不同的文件里,把一个大的数据文件拆分为多个小文件,还可以把这些小文件放在 ...

  7. 消息队列——RabbitMQ学习笔记

    消息队列--RabbitMQ学习笔记 1. 写在前面 昨天简单学习了一个消息队列项目--RabbitMQ,今天趁热打铁,将学到的东西记录下来. 学习的资料主要是官网给出的6个基本的消息发送/接收模型, ...

  8. 使用Python保存屏幕截图(不使用PIL)

    起因 在极客学院讲授<使用Python编写远程控制程序>的课程中,涉及到查看被控制电脑屏幕截图的功能. 如果使用PIL,这个需求只需要三行代码: from PIL import Image ...

  9. NodeJs之pm2

    pm2 pm2是一个进程管理工具,可以用它来管理你的node进程,并查看node进程的状态,当然也支持性能监控,进程守护,负载均衡等功能. 开发过程中建议时不时的参看官方详细命令行使用:命令行 pm2 ...

  10. iOS总结_UI层自我复习总结

    UI层复习笔记 在main文件中,UIApplicationMain函数一共做了三件事 根据第三个参数创建了一个应用程序对象 默认写nil,即创建的是UIApplication类型的对象,此对象看成是 ...

随机推荐

  1. iframe平铺到浏览器

    <!DOCTYPE html><html lang="en"><head> <meta charset="UTF-8" ...

  2. postgresql-9.0.18-1-linux.run启动

    下载地址:http://www.enterprisedb.com/products-services-training/pgdownload选择合适版本 基本都是下一步就可以到设置密码时记住密码以后要 ...

  3. mysql开发必知必会

    mysql的数据库的数据库,即存储mysql数据库的底层目录,是在/var/lib/mysql目录下(Linux,win在目录下的data中). 我们新创建的数据库db1就是在/var/lib/mys ...

  4. BZOJ题目(持续更新)

    bzoj1009:kmp想法+递推+矩阵快速幂.很好的想法,考虑用长串去kmp匹配短串,dp[i][j]表示匹配指针分别指在i.j位置时候,前i位母字符串一共有多少种可能性,那么dp[i][j]=Σd ...

  5. 笔记-迎难而上之Java基础进阶5

    Lambda表达式无参数无返回值的练习 //定义一个接口 public interface Cook{ public abstract void makeFood(); } public class ...

  6. Extjs grid增加或删除列后记住滚动条的位置

    IE下验证好使. { text: "Del", icon: 'Scripts/Ext/resources/images/icons/application_form_delete. ...

  7. Maven插件maven-antrun-plugin的使用

    以下引用官方的介绍http://maven.apache.org/plugins/maven-antrun-plugin/: 一.什么是maven-antrun-plugin? 该插件提供从Maven ...

  8. 关于在iOS设备上探测WIFI,3G,GPRS使用情况的细节

    由于设计的游戏需要有一些联网请求,但有时候在设备未连接网络的情况下,如果对网络情况不加以判断,则可能造成游戏为了等游戏超时,浪费不必要的时间. 所以在游戏启动时检测一下网络状况是很必要的,而且当玩家的 ...

  9. 【报错】项目启动部署时报错:java.lang.NoSuchMethodError

    报错: ================================================================================================ ...

  10. Activity的启动模式全解standard,singleTop,singleTask,singleInstance

    在android中控制Activity的启动模式的属性主要控制两大功能: 1,控制activity 进入哪一个任务task 中,   有两种可能,进入启动task中,进入指定taskAffinity的 ...