RabbitMQ Connector
https://ci.apache.org/projects/flink/flink-docs-master/dev/connectors/rabbitmq.html
RabbitMQ Source
This connector provides a RMQSource
class to consume messages from a RabbitMQ queue. This source provides three different levels of guarantees, depending on how it is configured with Flink:
- Exactly-once: In order to achieve exactly-once guarantees with the RabbitMQ source, the following is required -
- Enable checkpointing: With checkpointing enabled, messages are only acknowledged (hence, removed from the RabbitMQ queue) when checkpoints are completed.
- Use correlation ids: Correlation ids are a RabbitMQ application feature. You have to set it in the message properties when injecting messages into RabbitMQ. The correlation id is used by the source to deduplicate any messages that have been reprocessed when restoring from a checkpoint.
- Non-parallel source: The source must be non-parallel (parallelism set to 1) in order to achieve exactly-once. This limitation is mainly due to RabbitMQ’s approach to dispatching messages from a single queue to multiple consumers.
At-least-once: When checkpointing is enabled, but correlation ids are not used or the source is parallel, the source only provides at-least-once guarantees.
- No guarantee: If checkpointing isn’t enabled, the source does not have any strong delivery guarantees. Under this setting, instead of collaborating with Flink’s checkpointing, messages will be automatically acknowledged once the source receives and processes them.
RabbitMQ Connector的更多相关文章
- <译>Flink编程指南
Flink 的流数据 API 编程指南 Flink 的流数据处理程序是常规的程序 ,通过再流数据上,实现了各种转换 (比如 过滤, 更新中间状态, 定义窗口, 聚合).流数据可以来之多种数据源 (比如 ...
- Python 【第六章】:Python操作 RabbitMQ、Redis、Memcache、SQLAlchemy
Memcached Memcached 是一个高性能的分布式内存对象缓存系统,用于动态Web应用以减轻数据库负载.它通过在内存中缓存数据和对象来减少读取数据库的次数,从而提高动态.数据库驱动网站的速度 ...
- Python操作 RabbitMQ、Redis、Memcache、SQLAlchemy
Memcached Memcached 是一个高性能的分布式内存对象缓存系统,用于动态Web应用以减轻数据库负载.它通过在内存中缓存数据和对象来减少读取数据库的次数,从而提高动态.数据库驱动网站的速度 ...
- Python之路【第九篇】:Python操作 RabbitMQ、Redis、Memcache、SQLAlchemy
Python之路[第九篇]:Python操作 RabbitMQ.Redis.Memcache.SQLAlchemy Memcached Memcached 是一个高性能的分布式内存对象缓存系统,用 ...
- ActiveMQ, Qpid, HornetQ and RabbitMQ in Comparison
国内私募机构九鼎控股打造APP,来就送 20元现金领取地址:http://jdb.jiudingcapital.com/phone.html内部邀请码:C8E245J (不写邀请码,没有现金送)国内私 ...
- Day10 - Python协程、异步IO、redis缓存、rabbitMQ队列
Python之路,Day9 - 异步IO\数据库\队列\缓存 本节内容 Gevent协程 Select\Poll\Epoll异步IO与事件驱动 Python连接Mysql数据库操作 RabbitM ...
- Amqp整合com.rabbitmq.client.ShutdownSignalException: channel error; protocol method异常处理
java.io.IOException at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:126) at com.rabbitmq ...
- Memcache,redis,rabbitMQ,SQLAlchemy
Memcached Memcached 是一个高性能的分布式内存对象缓存系统,用于动态Web应用以减轻数据库负载.它通过在内存中缓存数据和对象来减少读取数据库的次数,从而提高动态.数据库驱动网站的速度 ...
- 各种缓存(Memcached、Redis、RabbitMQ、SQLlchemy)
Memcached Memcached 是一个高性能的分布式内存对象缓存系统,用于动态Web应用以减轻数据库负载.它通过在内存中缓存数据和对象来减少读取数据库的次数,从而提高动态.数据库驱动网站的速度 ...
随机推荐
- hdu3038 How Many Answers Are Wrong
TT and FF are ... friends. Uh... very very good friends -________-b FF is a bad boy, he is always wo ...
- 把项目变成intellij idea和eclipse项目
就通过maven把它build成一个IDE项目,执行以下命令,打开CMD: $ cd 项目名 $ mvn eclipse:eclipse or mvn idea:idea
- __new__ 与 __init__的区别
原文地址:http://www.cnblogs.com/ifantastic/p/3175735.html 单例模式 class Singleton(object): def __new__(cls, ...
- 区间翻转(codevs 3243)
题目描述 Description 给出N个数,要求做M次区间翻转(如1 2 3 4变成4 3 2 1),求出最后的序列 输入描述 Input Description 第一行一个数N,下一行N个数表示原 ...
- ajax 提交数组,mybatis 数组接受
function jinxingzhong() { // 配送单编号:psd2017090103 var divHtml = ""; $.ajax({ type: "PO ...
- CSV文件导出2
public void exportCSVFile( HttpServletResponse response, ResultSet rs,String fileName,String headers ...
- css3 画三角形
/*箭头向上*/ .arrow-up { width:0; height:0; border-left:20px solid transparent; border-right:20px solid ...
- CMDB资产管理系统的数据表设计
Server表: asset = models.OneToOneField('Asset') 主机名(hostname) sn号(sn) 制造商(manufacture) 系统(os_platform ...
- windows symbol server调试
linux下gdb强大的调试功能让人印象深刻,一直以为windows下调试可执行程序非常让人头痛.经一些高人指点后知道原来windows下还有symbol server这种调试工具 参见下面两个文档 ...
- JavaSE的static、final、abstract修饰符
static :静态常量,静态方法,静态代码块 静态变量: 静态变量属于类的,使用类名来访问,非静态变量是属于对象的,"必须"使用对象来访问. 注意: ...