What is the difference between message queue pattern and publish-subscribe? - Quora https://www.quora.com/What-is-the-difference-between-message-queue-pattern-and-publish-subscribe

A message queue is a stream of messages which typically so to exactly one consumer.

In Publish/Subscriber, you can have any number of subscribers (including zero) listening to the same messages.

A messaging system is a software interface, that maintains a stream of messages to transfer it from one application to another. This system maintains a queue in its in-memory or in its disk to store the messages that are produced by the producers. It will wipe out those messages once they are consumed by the consumers. It follows two kinds of patterns viz. message queueing pattern and pub-sub pattern.

Message queueing pattern is a kind of point-to-point messaging system, where the message from the queue will be wiped once it is consumed by any one of the consumer. It's similar to the default behavior of Post Office Protocol, where the message from the server will be deleted once it is consumed by any of the devices.

In Publish-Subscibe pattern, publishers are the set of producers that publishes different categories of messages and subscribers are set of consumers that consumes messages from the subscribed message categories. Unlike point-to-point messaging systems, the message will be wiped from the queue only if it is consumed by all of the subscribers. In certain messaging systems like kafka, there is a retention policy specifies how long the message should stay in the queue. So the message will be available in the queue for the specified duration even though it is consumed by all of the subscribers.

Kafka as a Messaging System

How does Kafka's notion of streams compare to a traditional enterprise messaging system?

Messaging traditionally has two models: queuing and publish-subscribe. In a queue, a pool of consumers may read from a server and each record goes to one of them; in publish-subscribe the record is broadcast to all consumers. Each of these two models has a strength and a weakness. The strength of queuing is that it allows you to divide up the processing of data over multiple consumer instances, which lets you scale your processing. Unfortunately, queues aren't multi-subscriber—once one process reads the data it's gone. Publish-subscribe allows you broadcast data to multiple processes, but has no way of scaling processing since every message goes to every subscriber.

The consumer group concept in Kafka generalizes these two concepts. As with a queue the consumer group allows you to divide up processing over a collection of processes (the members of the consumer group). As with publish-subscribe, Kafka allows you to broadcast messages to multiple consumer groups.

The advantage of Kafka's model is that every topic has both these properties—it can scale processing and is also multi-subscriber—there is no need to choose one or the other.

Kafka has stronger ordering guarantees than a traditional messaging system, too.

A traditional queue retains records in-order on the server, and if multiple consumers consume from the queue then the server hands out records in the order they are stored. However, although the server hands out records in order, the records are delivered asynchronously to consumers, so they may arrive out of order on different consumers. This effectively means the ordering of the records is lost in the presence of parallel consumption. Messaging systems often work around this by having a notion of "exclusive consumer" that allows only one process to consume from a queue, but of course this means that there is no parallelism in processing.

Kafka does it better. By having a notion of parallelism—the partition—within the topics, Kafka is able to provide both ordering guarantees and load balancing over a pool of consumer processes. This is achieved by assigning the partitions in the topic to the consumers in the consumer group so that each partition is consumed by exactly one consumer in the group. By doing this we ensure that the consumer is the only reader of that partition and consumes the data in order. Since there are many partitions this still balances the load over many consumer instances. Note however that there cannot be more consumer instances in a consumer group than partitions.

Apache Kafka http://kafka.apache.org/intro#kafka_mq

What is the difference between message queue pattern and publish-subscribe?的更多相关文章

  1. 消息队列(Message Queue)基本概念(转)

    背景 之前做日志收集模块时,用到flume.另外也有的方案,集成kafaka来提升系统可扩展性,其中涉及到消息队列当时自己并不清楚为什么要使用消息队列.而在我自己提出的原始日志采集方案中不适用消息队列 ...

  2. 快速比较 Kafka 与 Message Queue 的区别

    https://hackernoon.com/a-super-quick-comparison-between-kafka-and-message-queues-e69742d855a8 A supe ...

  3. 什么是 Message Queue

    Message Queue 是一种非同步的从一个服务到另一个服务的交流形式, 被用于无服务器架构和微服务架构中. Messages 被储存在一个队列中直到被处理了或被删除. 每个Messages只会被 ...

  4. Top 10 Uses For A Message Queue

    We’ve been working with, building, and evangelising message queues for the last year, and it’s no se ...

  5. You Probably Don’t Need a Message Queue

    原文地址 I’m a minimalist, and I don’t like to complicate software too early and unnecessarily. And addi ...

  6. 初识Message Queue之--基础篇

    之前我在项目中要用到消息队列相关的技术时,一直让Redis兼职消息队列功能,一个偶然的机会接触到了MSMQ消息队列.秉着技术还是专业的好为原则,对MSMQ进行了学习,以下是我个人的学习笔记. 一.什么 ...

  7. MSMQ(Microsoft Message Queue)

    http://www.cnblogs.com/sk-net/archive/2011/11/25/2232341.html 利用 MSMQ(Microsoft Message Queue),应用程序开 ...

  8. Message Queue vs. Web Services?

    From stackoverflow.com When you use a web service you have a client and a server: If the server fail ...

  9. hdu 1509 Windows Message Queue

    题目连接 http://acm.hdu.edu.cn/showproblem.php?pid=1509 Windows Message Queue Description Message queue ...

随机推荐

  1. jenkins之Tomcat7+jdk1.7+jenkins

    目的 在开发中,需要经常频繁的对测试服务器进行部署,而且在多人协同中开发经常遇到的问题就是别人更新了他的代码,而你去更新你的代码时并没有更新到别人的代码,导致测试环境的代码不是最新,当然这个问题也好解 ...

  2. redis2.3.7安装时出现undefined reference to `clock_gettime'

    (转自:http://blog.csdn.net/qq_28779503/article/details/54844988) undefined reference to `clock_gettime ...

  3. [BZOJ1663] [Usaco2006 Open]赶集(spfa最长路)

    传送门 按照时间t排序 如果 t[i] + map[i][j] <= t[j],就在i和j之间连一条边 然后spfa找最长路 #include <queue> #include &l ...

  4. [BZOJ1583] [Usaco2009 Mar]Moon Mooing 哞哞叫(队列)

    传送门 思想有点像蚯蚓那个题 #include <cstdio> #define N 4000001 #define LL long long #define min(x, y) ((x) ...

  5. 648. Replace Words

    Problem statement In English, we have a concept called root, which can be followed by some other wor ...

  6. 【强化学习】百度云BCC openai gym 环境配置

    openai gym是RL算法的测试床.为了直观和便于调试,我们需要可视化界面.然而远程服务器使用openGL的问题比较棘手,这篇博客记录在百度云BCC上配置openai gym环境的注意事项. 按照 ...

  7. 【shell】shell编程(一)-入门

    如今,不会Linux的程序员都不意思说自己是程序员,而不会shell编程就不能说自己会Linux.说起来似乎shell编程很屌啊,然而不用担心,其实shell编程真的很简单.背景 什么是shell编程 ...

  8. Scrapy学习-4-Items类&Pipelines类

    items类使用 作用 能使得我们非常方便的操作字段名 在items.py中定制我们的类 class ArticleItem(scrapy.Item): title = scrapy.Field() ...

  9. jQuery插件封装系列(一)—— 金额录入框

    基于jQuery原型封装数值录入框,禁止录入.粘贴非数值字符 (function ($) { // 数值输入框 $.fn.numbox = function (options) { var type ...

  10. android 完美退出应用程序。

    Android 程序在点击回退键时,如果只有一个activity,调用finish()方法就能退出界面,如果有多个界面,在调用该方法时,只会销毁当前的activity,显示栈顶的其它activity, ...