一、ZMQ 是什么

　　阅读了 ZMQ 的 Guide 文档后，我的理解是，这是个类似于 Socket 的一系列接口，他跟 Socket 的区别是：普通的 socket 是端到端的（1:1的关系），而 ZMQ 却是可以N：M 的关系，人们对 BSD 套接字的了解较多的是点对点的连接，点对点连接需要显式地建立连接、销毁连接、选择协议（TCP/UDP）和处理错误等，而 ZMQ 屏蔽了这些细节，让你的网络编程更为简单。ZMQ 用于 node 与 node 间的通信，node 可以是主机或者是进程。

二、本文的目的

　　在集群对外提供服务的过程中，我们有很多的配置，需要根据需要随时更新，那么这个信息如何推动到各个节点？并且保证信息的一致性和可靠性？本文在介绍 ZMQ 基本理论的基础上，试图使用 ZMQ 实现一个配置分发中心。从一个节点，将信息无误的分发到各个服务器节点上，并保证信息正确性和一致性。

三、ZMQ 的三个基本模型

1.请求回应模型。由请求端发起请求，并等待回应端回应请求。从请求端来看，一定是一对对收发配对的；
反之，在回应端一定是发收对。请求端和回应端都可以是1：N的模型。通常把1认为是server，N认为是Client。
0MQ可以很好的支持路由功能（实现路由功能的组件叫做Device），把1：N扩展为N：M（只需要加入若干路由节点）。
从这个模型看，更底层的端点地址是对上层隐藏的。每个请求都隐含回应地址，而应用则不关心它。

2.发布订阅模型。这个模型里，发布端是单向只发送数据的，且不关心是否把全部的信息都发送给订阅者。
如果发布端开始发布信息的时候，订阅端尚未连接上，这些信息直接丢弃。
不过一旦订阅端连接上来，中间会保证没有信息丢失。
同样，订阅端则只负责接收，而不能反馈。
如果发布端和订阅端需要交互（比如要确认订阅者是否已经连接上），则使用额外的socket采用请求回应模型满足这个需求。

3.管道模型。这个模型里，管道是单向的，从PUSH端单向的向PULL端单向的推送数据流。

ZMQ的请求答复模型

ZMQ 的 hello world！

由 Client 发起请求，并等待 Server 回应请求。请求端发送一个简单的 hello，服务端则回应一个 world

request.py

 # !/usr/bin/python

 # coding=utf-8

 #

 # Hello World client in Python

 # Connects REQ socket to tcp://localhost:5555

 # Sends "Hello" to server, expects "World" back 

 import zmq

 context = zmq.Context()

 # Socket to talk to server

 print("Connecting to hello world server..." )

 socket = context.socket(zmq.REQ)

 socket.connect("tcp://localhost:5555") 

 # Do 10 requests, waiting each time for a response

 for request in range (1,10):

     print("Sending request %d ..." % request)

     socket.send(b"Hello")

     # Get the reply.

     message = socket.recv()

     print("Received reply [%s]" % message)
lxy@lenovo-pc:~/code/python/zmq$ python3 request.py

Connecting to hello world server...

Sending request 1 ...

Received reply [b'World']

Sending request 2 ...

Received reply [b'World']

Sending request 3 ...

Received reply [b'World']

Sending request 4 ...

Received reply [b'World']

Sending request 5 ...

Received reply [b'World']

Sending request 6 ...

Received reply [b'World']

Sending request 7 ...

Received reply [b'World']

Sending request 8 ...

Received reply [b'World']

Sending request 9 ...

Received reply [b'World']

reply.py

 # !/usr/bin/python

 # coding=utf-8

 #

 # Hello World server in Python

 # Binds REP socket to tcp://*:5555

 # Expects "Hello" from client, replies with "World" 

 import zmq

 import time

 context = zmq.Context()

 socket = context.socket(zmq.REP)

 socket.bind("tcp://*:5555")

 while True:

     message = socket.recv()

     print("Received request: %s" % message)

     #time.sleep(1)

     socket.send_string("World")
lxy@lenovo-pc:~/code/python/zmq$ python3 reply.py

Received request: b'Hello'

Received request: b'Hello'

Received request: b'Hello'

Received request: b'Hello'

Received request: b'Hello'

Received request: b'Hello'

Received request: b'Hello'

Received request: b'Hello'

Received request: b'Hello'

ZMQ的发布订阅模型

一个广播server为现场足球赛

publish.py

 # !/usr/bin/python

 # coding=utf-8

 import zmq

 from random import choice

 context = zmq.Context()

 socket = context.socket(zmq.PUB)

 socket.bind("tcp://127.0.0.1:5000")

 countries = ["netherlands","brazil","germany","portugal"]

 events = ['yellow card', 'red card', 'goal', 'corner', 'foul'] 

 while True:

     msg = choice(countries)+" "+choice(events)

     print("-> %s" % msg)

     socket.send(msg.encode(encoding='utf-8'))
-> portugal corner

-> portugal yellow card

-> portugal goal

-> netherlands yellow card

-> germany yellow card

-> brazil yellow card

-> portugal goal

-> germany corner

subscribe.py

 # !/usr/bin/python

 # coding=utf-8

 import zmq 

 context = zmq.Context()

 socket = context.socket(zmq.SUB)

 socket.connect("tcp://127.0.0.1:5000")

 socket.setsockopt(zmq.SUBSCRIBE, b"netherlands")

 socket.setsockopt(zmq.SUBSCRIBE, b"germany") 

 while True:

     print(socket.recv())
netherlands red card

netherlands goal

netherlands red card

germany foul

netherlands yellow card

germany foul

netherlands goal

netherlands corner

germany foul

netherlands corner

ZMQ的管道模型

ventilator.py

 # !/usr/bin/python

 # coding=utf-8

 # Task ventilator

 # Binds PUSH socket to tcp://localhost:5557

 # Sends batch of tasks to workers via that socket

 import time

 import zmq 

 context = zmq.Context()

 socket = context.socket(zmq.PUSH)

 #print(dir(socket))

 #exit(0)

 socket.bind("tcp://127.0.0.1:5557") 

 input("Press Enter when the workers are ready: ")

 print("Sending tasks to workers ... \n")

 # send 100 tasks

 for task_nbr in range(1, 101):

     socket.send_string("task %d" % task_nbr)

 time.sleep(1)

worker.py

 # !/usr/bin/python

 # coding=utf-8

 #Task worker

 #Connects PULL socket to tcp://localhost:5557

 #Collects workloads from ventilator via that socket

 #Connects PUSH socket to tcp://localhost:5558

 #Sends results to sink via that socket

 import time

 import zmq

 context = zmq.Context() 

 receiver = context.socket(zmq.PULL)

 receiver.connect("tcp://127.0.0.1:5557") 

 sender = context.socket(zmq.PUSH)

 sender.connect("tcp://127.0.0.1:5558")

 while True:

     str = receiver.recv()

     print("Received reply : %s" % str)

     sender.send(str)

启动3个worker，输出结果如下

worker1

Received reply : b'task 1'

Received reply : b'task 4'

Received reply : b'task 7'

Received reply : b'task 10'

Received reply : b'task 13'

Received reply : b'task 16'

Received reply : b'task 19'

Received reply : b'task 22'

Received reply : b'task 25'

Received reply : b'task 28'

Received reply : b'task 31'

Received reply : b'task 34'

Received reply : b'task 37'

Received reply : b'task 40'

Received reply : b'task 43'

Received reply : b'task 46'

Received reply : b'task 49'

Received reply : b'task 52'

Received reply : b'task 55'

Received reply : b'task 58'

Received reply : b'task 61'

Received reply : b'task 64'

Received reply : b'task 67'

Received reply : b'task 70'

Received reply : b'task 73'

Received reply : b'task 76'

Received reply : b'task 79'

Received reply : b'task 82'

Received reply : b'task 85'

Received reply : b'task 88'

Received reply : b'task 91'

Received reply : b'task 94'

Received reply : b'task 97'

Received reply : b'task 100'

worker2

Received reply : b'task 2'

Received reply : b'task 5'

Received reply : b'task 8'

Received reply : b'task 11'

Received reply : b'task 14'

Received reply : b'task 17'

Received reply : b'task 20'

Received reply : b'task 23'

Received reply : b'task 26'

Received reply : b'task 29'

Received reply : b'task 32'

Received reply : b'task 35'

Received reply : b'task 38'

Received reply : b'task 41'

Received reply : b'task 44'

Received reply : b'task 47'

Received reply : b'task 50'

Received reply : b'task 53'

Received reply : b'task 56'

Received reply : b'task 59'

Received reply : b'task 62'

Received reply : b'task 65'

Received reply : b'task 68'

Received reply : b'task 71'

Received reply : b'task 74'

Received reply : b'task 77'

Received reply : b'task 80'

Received reply : b'task 83'

Received reply : b'task 86'

Received reply : b'task 89'

Received reply : b'task 92'

Received reply : b'task 95'

Received reply : b'task 98'

worker3

Received reply : b'task 3'

Received reply : b'task 6'

Received reply : b'task 9'

Received reply : b'task 12'

Received reply : b'task 15'

Received reply : b'task 18'

Received reply : b'task 21'

Received reply : b'task 24'

Received reply : b'task 27'

Received reply : b'task 30'

Received reply : b'task 33'

Received reply : b'task 36'

Received reply : b'task 39'

Received reply : b'task 42'

Received reply : b'task 45'

Received reply : b'task 48'

Received reply : b'task 51'

Received reply : b'task 54'

Received reply : b'task 57'

Received reply : b'task 60'

Received reply : b'task 63'

Received reply : b'task 66'

Received reply : b'task 69'

Received reply : b'task 72'

Received reply : b'task 75'

Received reply : b'task 78'

Received reply : b'task 81'

Received reply : b'task 84'

Received reply : b'task 87'

Received reply : b'task 90'

Received reply : b'task 93'

Received reply : b'task 96'

Received reply : b'task 99'

sink.py

 # !/usr/bin/python

 # coding=utf-8

 import time

 import zmq

 context = zmq.Context()

 receiver = context.socket(zmq.PULL)

 receiver.bind("tcp://127.0.0.1:5558")

 while True:

     str = receiver.recv()

     print("received: %s" % str)
received: b'task 1'

received: b'task 4'

received: b'task 7'

received: b'task 10'

received: b'task 13'

received: b'task 16'

received: b'task 19'

received: b'task 22'

received: b'task 25'

received: b'task 28'

received: b'task 31'

received: b'task 34'

received: b'task 37'

received: b'task 40'

received: b'task 43'

received: b'task 46'

received: b'task 49'

received: b'task 52'

received: b'task 55'

received: b'task 58'

received: b'task 61'

received: b'task 64'

received: b'task 67'

received: b'task 70'

received: b'task 73'

received: b'task 76'

received: b'task 79'

received: b'task 82'

received: b'task 85'

received: b'task 88'

received: b'task 91'

received: b'task 94'

received: b'task 97'

received: b'task 100'

received: b'task 2'

received: b'task 5'

received: b'task 8'

received: b'task 11'

received: b'task 14'

received: b'task 17'

received: b'task 20'

received: b'task 23'

received: b'task 26'

received: b'task 29'

received: b'task 32'

received: b'task 35'

received: b'task 38'

received: b'task 41'

received: b'task 44'

received: b'task 47'

received: b'task 50'

received: b'task 53'

received: b'task 56'

received: b'task 59'

received: b'task 62'

received: b'task 65'

received: b'task 68'

received: b'task 71'

received: b'task 74'

received: b'task 77'

received: b'task 80'

received: b'task 83'

received: b'task 86'

received: b'task 89'

received: b'task 92'

received: b'task 95'

received: b'task 98'

received: b'task 3'

received: b'task 6'

received: b'task 9'

received: b'task 12'

received: b'task 15'

received: b'task 18'

received: b'task 21'

received: b'task 24'

received: b'task 27'

received: b'task 30'

received: b'task 33'

received: b'task 36'

received: b'task 39'

received: b'task 42'

received: b'task 45'

received: b'task 48'

received: b'task 51'

received: b'task 54'

received: b'task 57'

received: b'task 60'

received: b'task 63'

received: b'task 66'

received: b'task 69'

received: b'task 72'

received: b'task 75'

received: b'task 78'

received: b'task 81'

received: b'task 84'

received: b'task 87'

received: b'task 90'

received: b'task 93'

received: b'task 96'

received: b'task 99'

从上面的输出可以看出，ventilator分配的100个任务被平均分配到了3个worker，最后由sink汇总

四、其他扩展模式

　　通常，一个节点，即可以作为 Server，同时也能作为 Client，通过 PipeLine 模型中的 Worker，他向上连接着任务分发，向下连接着结果搜集的 Sink 机器。因此，我们可以借助这种特性，丰富的扩展原有的三种模式。例如，一个代理 Publisher，作为一个内网的 Subscriber 接受信息，同时将信息，转发到外网，其结构图如图 4 所示。

五、多个服务器

ZMQ 和 Socket 的区别在于，前者支持N：M的连接，而后者则只是1：1的连接，那么一个 Client 连接多个 Server 的情况是怎样的呢，我们通过图 5 来说明。

server1.py

 import zmq

 context = zmq.Context()

 socket = context.socket(zmq.REP)

 socket.bind("tcp://127.0.0.1:5000") 

 while True:

     msg = socket.recv()

     print "Got", msg

     socket.send(msg)

server2.py

 import zmq

 context = zmq.Context()

 socket = context.socket(zmq.REP)

 socket.bind("tcp://127.0.0.1:6000") 

 while True:

     msg = socket.recv()

     print "Got", msg

     socket.send(msg)

client.py

 import zmq

 context = zmq.Context()

 socket = context.socket(zmq.REQ)

 socket.connect("tcp://127.0.0.1:5000")

 socket.connect("tcp://127.0.0.1:6000") 

 for i in range(10):

      msg = "msg %s" % i

      socket.send(msg)

      print "Sending", msg

      msg_in = socket.recv()

会发现client的请求会被均衡的分配给两个server

Example client output:

Sending msg 0

Sending msg 1

Sending msg 2

Sending msg 3

Sending msg 4

Sending msg 5

Sending msg 6

Sending msg 7

Sending msg 8

Sending msg 9

Example output server 1 at port 5000:

Got msg 0

Got msg 2

Got msg 4

Got msg 6

Got msg 8

Example output server 2 at port 6000:

Got msg 1

Got msg 3

Got msg 5

Got msg 7

Got msg 9

ZeroMQ，史上最快的消息队列的更多相关文章

  1. zeromq的安装，部署（号称最快的消息队列，消息中间件）

    1:Storm作为一个实时处理的框架,产生的消息需要快速的进行处理,比如存在消息队列ZeroMQ里面. 由于消息队列ZeroMQ是C++写的,而我们的程序是运行在JVM虚拟机里面的.所以需要jzmq这 ...

  2. RabbitMQ,Apache的ActiveMQ,阿里RocketMQ,Kafka,ZeroMQ,MetaMQ,Redis也可实现消息队列，RabbitMQ的应用场景以及基本原理介绍，RabbitMQ基础知识详解，RabbitMQ布曙

    消息队列及常见消息队列介绍 2017-10-10 09:35操作系统/客户端/人脸识别 一.消息队列(MQ)概述 消息队列(Message Queue),是分布式系统中重要的组件,其通用的使用场景可以 ...

  3. 全解史上最快的JOSN解析库 - alibaba Fastjson

    JSON,全称:JavaScript Object Notation,作为一个常见的轻量级的数据交换格式,应该在一个程序员的开发生涯中是常接触的.简洁和清晰的层次结构使得 JSON 成为理想的数据交换 ...

  4. 谁才是最快的消息队列:ActiveMQ, RabbitMQ[转]

    Lately I performed a message queue benchmark, comparing several queuing frameworks (RabbitMQ, Active ...

  5. rabbitmq学习（九） —— 关于消息队列的选型

    转自http://cmsblogs.com/?p=3846 在IM这种讲究高并发.高消息吞吐的互联网场景下,MQ消息中间件是个很重要的基础设施,它在IM系统的服务端架构中担当消息中转.消息削峰.消息交 ...

  6. 消息队列（message queue）

    最近纠结于一个问题,就是horizon 在处理前台数据的时候非得等到cinder client将数据全部获取后才开始执行horizon的下一行代码,这给大量数据显示造成了很大的时延,其实对于用户体验来 ...

  7. 网站架构：消息队列 Java后端架构

    2017-01-13  一.消息队列概述 消息队列中间件是分布式系统中重要的组件,主要解决应用耦合,异步消息,流量削锋等问题.实现高性能,高可用,可伸缩和最终一致性架构.是大型分布式系统不可缺少的中间 ...

  8. IM开发基础知识补课(五)：通俗易懂，正确理解并用好MQ消息队列

    1.引言 消息是互联网信息的一种表现形式,是人利用计算机进行信息传递的有效载体,比如即时通讯网坛友最熟悉的即时通讯消息就是其具体的表现形式之一. 消息从发送者到接收者的典型传递方式有两种: 1)一种我 ...

  9. 消息队列ZeroMQ

    消息队列概念 消息队列技术是分布式应用间交换信息的一种技术.消息队列可以驻留在内存或者磁盘上,队列存储消息直到它们被应用程序读走.通过消息队列,应用程序可以独立的执行,它们不需要知道彼此的位置,或者在 ...

随机推荐

  1. android仿网易云音乐引导页、仿书旗小说Flutter版、ViewPager切换、爆炸菜单、风扇叶片效果等源码

    Android精选源码 复现网易云音乐引导页效果 高仿书旗小说 Flutter版,支持iOS.Android Android Srt和Ass字幕解析器 Material Design ViewPage ...

  2. list循环 字典循环 字符串常用方法

    list = ['xiaoli','xiaohua','huali']user = {'zhang':'123','lin':'321','chen':'222'}#list循环for stu in ...

  3. 7.学完linux系统运维到底可以做什么？

    linux运维到底可以做什么?(略有改动原文.排版) 运维,很容易从字面理解为运营.维护. 很多朋友认为,在互联网公司中linux系统运维的工作就是安装系统,部署服务.处理紧急故障,为公司里的开发人员 ...

  4. Multiple alleles|an intuitive argument|

    I.5 Multiple alleles. 由两个等位基因拓展到多个等位基因,可以得到更多种二倍体基因型: 所以单个等位基因的概率(用i代指某个基因,pi*是该基因的频率)是(以计数的方法表示) 所以 ...

  5. [一般图最大匹配]Bimatching

    10566 Bimatching 题意:一个男生必须跟两个女生匹配,求最大匹配 思路:一般的二分图匹配做不了,网络流也不会建图,这题采用的是一般图匹配 首先在原来二分图的基础上,将一个男生拆成两个点 ...

  6. MySQL_学习资料

    https://mp.weixin.qq.com/s/qOmyaEEpVJTUMZYfomp3ug

  7. s01字符串---蓝桥杯

    问题描述 s01串初始为"0" 按以下方式变换 0变1,1变01 输入格式 1个整数(0~19) 输出格式 n次变换后s01串 样例输入 3 样例输出 101 数据规模和约定 0~ ...

  8. 监控 Linux 服务器活动的几个命令(watch top ac)

    watch.top 和 ac 命令为我们监视 Linux 服务器上的活动提供了一些十分高效的途径. 为了在获取系统活动时更加轻松,Linux 系统提供了一系列相关的命令.在这篇文章中,我们就一起来看看 ...

  9. 吴裕雄--天生自然python学习笔记：python 文件批量查找

    在多个文本文件中查找 我们首先来学习文本文件的查找字符 . 我们通过 os.walk 扩大查找范围, 查找指定目录和子目录下的文件. 应用程序总览 读取 当 前目录及子目录下的所有 PY 和 txt ...

  10. q检验|新复极差法|LSD|二因素方差分析

    生物统计与实验设计 放大程度q检验:精度较高>新复极差法:各种错误比较平均>LSD 其中,LSD不随M的变化而变化,但是SSR和q-test会随M变化而变化. 第一步代表了方差分析的核心思 ...