Zookeeper-watcher机制源码分析(一)
Watcher的基本流程
ZooKeeper 的 Watcher 机制,总的来说可以分为三个过程:客户端注册 Watcher、服务器处理 Watcher 和客户端回调 Watcher
客户端注册watcher有3种方式,getData、exists、getChildren;以如下代码为例来分析整个触发机制的原理
ZooKeeper zookeeper=new ZooKeeper(“192.168.11.152:2181”,4000,new Watcher(){ public void processor(WatchedEvent event){ System.out.println(“event.type”); } }); zookeeper.create(“/mic”,”0”.getByte(),ZooDefs.Ids. OPEN_ACL_UNSAFE,CreateModel. PERSISTENT); //创建节点 zookeeper.exists(“/mic”,true); //注册监听 zookeeper.setData(“/mic”, “1”.getByte(),-1) ; //修改节点的值触发监听 |
ZooKeeper API的初始化过程
ZooKeeper zookeeper=new ZooKeeper(“192.168.11.152:2181”,4000,new Watcher(){ public void processor(WatchedEvent event){ System.out.println(“event.type”); } }); |
在创建一个 ZooKeeper 客户端对象实例时,我们通过new Watcher()向构造方法中传入一个默认的 Watcher, 这个 Watcher 将作为整个 ZooKeeper会话期间的默认 Watcher,会一直被保存在客户端 ZKWatchManager 的 defaultWatcher 中;代码如下
public ZooKeeper(String connectString, int sessionTimeout, Watcher watcher, boolean canBeReadOnly, HostProvider aHostProvider, ZKClientConfig clientConfig) throws IOException { LOG.info("Initiating client connection, connectString=" + connectString + " sessionTimeout=" + sessionTimeout + " watcher=" + watcher); if (clientConfig == null) { clientConfig = new ZKClientConfig(); } this.clientConfig = clientConfig; watchManager = defaultWatchManager(); watchManager.defaultWatcher = watcher; --在这里将watcher设置到ZKWatchManager ConnectStringParser connectStringParser = new ConnectStringParser( connectString); hostProvider = aHostProvider; --初始化了ClientCnxn,并且调用cnxn.start()方法 cnxn = new ClientCnxn(connectStringParser.getChrootPath(), hostProvider, sessionTimeout, this, watchManager, getClientCnxnSocket(), canBeReadOnly); cnxn.start(); } |
ClientCnxn:是Zookeeper客户端和Zookeeper服务器端进行通信和事件通知处理的主要类,它内部包含两个类,
1. SendThread :负责客户端和服务器端的数据通信, 也包括事件信息的传输
2. EventThread : 主要在客户端回调注册的Watchers进行通知处理
ClientCnxn初始化
public ClientCnxn(String chrootPath, HostProvider hostProvider, int sessionTimeout, ZooKeeper zooKeeper, ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket, long sessionId, byte[] sessionPasswd, boolean canBeReadOnly) { this.zooKeeper = zooKeeper; this.watcher = watcher; this.sessionId = sessionId; this.sessionPasswd = sessionPasswd; this.sessionTimeout = sessionTimeout; this.hostProvider = hostProvider; this.chrootPath = chrootPath; connectTimeout = sessionTimeout / hostProvider.size(); readTimeout = sessionTimeout * 2 / 3; readOnly = canBeReadOnly; sendThread = new SendThread(clientCnxnSocket); --初始化sendThread eventThread = new EventThread(); --初始化eventThread this.clientConfig=zooKeeper.getClientConfig(); } public void start() { --启动两个线程 sendThread.start(); eventThread.start(); } |
客户端通过exists注册监听
zookeeper.exists(“/mic”,true); //注册监听 |
通过exists方法来注册监听,代码如下
public Stat exists(final String path, Watcher watcher) throws KeeperException, InterruptedException { final String clientPath = path; PathUtils.validatePath(clientPath); // the watch contains the un-chroot path WatchRegistration wcb = null; if (watcher != null) { wcb = new ExistsWatchRegistration(watcher, clientPath); //构建ExistWatchRegistration } final String serverPath = prependChroot(clientPath); RequestHeader h = new RequestHeader(); h.setType(ZooDefs.OpCode.exists); //设置操作类型为exists ExistsRequest request = new ExistsRequest(); // 构造ExistsRequest request.setPath(serverPath); request.setWatch(watcher != null); //是否注册监听 SetDataResponse response = new SetDataResponse(); //设置服务端响应的接收类 //将封装的RequestHeader、ExistsRequest、SetDataResponse、WatchRegistration添加到发送队列 ReplyHeader r = cnxn.submitRequest(h, request, response, wcb); if (r.getErr() != 0) { if (r.getErr() == KeeperException.Code.NONODE.intValue()) { return null; } throw KeeperException.create(KeeperException.Code.get(r.getErr()), clientPath); } //返回exists得到的结果(Stat信息) return response.getStat().getCzxid() == -1 ? null : response.getStat(); } |
cnxn.submitRequest
public ReplyHeader submitRequest(RequestHeader h, Record request, Record response, WatchRegistration watchRegistration, WatchDeregistration watchDeregistration) throws InterruptedException { ReplyHeader r = new ReplyHeader(); //将消息添加到队列,并构造一个Packet传输对象 Packet packet = queuePacket(h, r, request, response, null, null, null, null, watchRegistration, watchDeregistration); synchronized (packet) { while (!packet.finished) { //在数据包没有处理完成之前,一直阻塞 packet.wait(); } } return r; } |
public Packet queuePacket(RequestHeader h, ReplyHeader r, Record request, Record response, AsyncCallback cb, String clientPath, String serverPath, Object ctx, WatchRegistration watchRegistration, WatchDeregistration watchDeregistration) { //将相关传输对象转化成Packet Packet packet = null; packet = new Packet(h, r, request, response, watchRegistration); packet.cb = cb; packet.ctx = ctx; packet.clientPath = clientPath; packet.serverPath = serverPath; packet.watchDeregistration = watchDeregistration; synchronized (state) { if (!state.isAlive() || closing) { conLossPacket(packet); } else { if (h.getType() == OpCode.closeSession) { closing = true; } outgoingQueue.add(packet); //添加到outgoingQueue } } sendThread.getClientCnxnSocket().packetAdded();//此处是多路复用机制,唤醒Selector,告诉他有数据包添加过来了 return packet; } |
在 ZooKeeper 中,Packet 是一个最小的通信协议单元,即数据包。Pakcet 用于进行客户端与服务端之间的网络传输,任何需要传输的对象都需要包装成一个 Packet 对象。在 ClientCnxn 中 WatchRegistration 也会被封装到 Pakcet 中,然后由 SendThread 线程调用queuePacket方法把 Packet 放入发送队列中等待客户端发送,这又是一个异步过程,分布式系统采用异步通信是一个非常常见的手段
SendThread的发送过程
在初始化连接的时候,zookeeper初始化了两个线程并且启动了。接下来我们来分析SendThread的发送过程,因为是一个线程,所以启动的时候会调用SendThread.run方法
public void run() { clientCnxnSocket.introduce(this, sessionId, outgoingQueue); clientCnxnSocket.updateNow(); clientCnxnSocket.updateLastSendAndHeard(); int to; long lastPingRwServer = Time.currentElapsedTime(); final int MAX_SEND_PING_INTERVAL = 10000; //10 seconds while (state.isAlive()) { try { if (!clientCnxnSocket.isConnected()) {// 如果没有连接:发起连接 // don't re-establish connection if we are closing if (closing) { break; } startConnect(); //发起连接 clientCnxnSocket.updateLastSendAndHeard(); } if (state.isConnected()) { //如果是连接状态,则处理sasl的认证授权 // determine whether we need to send an AuthFailed event. if (zooKeeperSaslClient != null) { boolean sendAuthEvent = false; if (zooKeeperSaslClient.getSaslState() == ZooKeeperSaslClient.SaslState.INITIAL) { try { zooKeeperSaslClient.initialize(ClientCnxn.this); } catch (SaslException e) { LOG.error("SASL authentication with Zookeeper Quorum member failed: " + e); state = States.AUTH_FAILED; sendAuthEvent = true; } } KeeperState authState = zooKeeperSaslClient.getKeeperState(); if (authState != null) { if (authState == KeeperState.AuthFailed) { // An authentication error occurred during authentication with the Zookeeper Server. state = States.AUTH_FAILED; sendAuthEvent = true; } else { if (authState == KeeperState.SaslAuthenticated) { sendAuthEvent = true; } } } if (sendAuthEvent == true) { eventThread.queueEvent(new WatchedEvent( Watcher.Event.EventType.None, authState,null)); } } to = readTimeout - clientCnxnSocket.getIdleRecv(); } else { to = connectTimeout - clientCnxnSocket.getIdleRecv(); } //to,表示客户端距离timeout还剩多少时间,准备发起ping连接 if (to <= 0) {//表示已经超时了。 String warnInfo; warnInfo = "Client session timed out, have not heard from server in " + clientCnxnSocket.getIdleRecv() + "ms" + " for sessionid 0x" + Long.toHexString(sessionId); LOG.warn(warnInfo); throw new SessionTimeoutException(warnInfo); } if (state.isConnected()) { //计算下一次ping请求的时间 int timeToNextPing = readTimeout / 2 - clientCnxnSocket.getIdleSend() - ((clientCnxnSocket.getIdleSend() > 1000) ? 1000 : 0); //send a ping request either time is due or no packet sent out within MAX_SEND_PING_INTERVAL if (timeToNextPing <= 0 || clientCnxnSocket.getIdleSend() > MAX_SEND_PING_INTERVAL) { sendPing(); //发送ping请求 clientCnxnSocket.updateLastSend(); } else { if (timeToNextPing < to) { to = timeToNextPing; } } } // If we are in read-only mode, seek for read/write server if (state == States.CONNECTEDREADONLY) { long now = Time.currentElapsedTime(); int idlePingRwServer = (int) (now - lastPingRwServer); if (idlePingRwServer >= pingRwTimeout) { lastPingRwServer = now; idlePingRwServer = 0; pingRwTimeout = Math.min(2*pingRwTimeout, maxPingRwTimeout); pingRwServer(); } to = Math.min(to, pingRwTimeout - idlePingRwServer); } 调用clientCnxnSocket,发起传输 其中 pendingQueue是一个用来存放已经发送、等待回应的Packet队列, clientCnxnSocket默认使用ClientCnxnSocketNIO(ps:还记得在哪里初始化吗?在实例化zookeeper的时候) clientCnxnSocket.doTransport(to, pendingQueue, ClientCnxn.this); } catch (Throwable e) { if (closing) { if (LOG.isDebugEnabled()) { // closing so this is expected LOG.debug("An exception was thrown while closing send thread for session 0x" + Long.toHexString(getSessionId()) + " : " + e.getMessage()); } break; } else { // this is ugly, you have a better way speak up if (e instanceof SessionExpiredException) { LOG.info(e.getMessage() + ", closing socket connection"); } else if (e instanceof SessionTimeoutException) { LOG.info(e.getMessage() + RETRY_CONN_MSG); } else if (e instanceof EndOfStreamException) { LOG.info(e.getMessage() + RETRY_CONN_MSG); } else if (e instanceof RWServerFoundException) { LOG.info(e.getMessage()); } else { LOG.warn( "Session 0x" + Long.toHexString(getSessionId()) + " for server " + clientCnxnSocket.getRemoteSocketAddress() + ", unexpected error" + RETRY_CONN_MSG, e); } // At this point, there might still be new packets appended to outgoingQueue. // they will be handled in next connection or cleared up if closed. cleanup(); if (state.isAlive()) { eventThread.queueEvent(new WatchedEvent( Event.EventType.None, Event.KeeperState.Disconnected, null)); } clientCnxnSocket.updateNow(); clientCnxnSocket.updateLastSendAndHeard(); } } } synchronized (state) { // When it comes to this point, it guarantees that later queued // packet to outgoingQueue will be notified of death. cleanup(); } clientCnxnSocket.close(); if (state.isAlive()) { eventThread.queueEvent(new WatchedEvent(Event.EventType.None, Event.KeeperState.Disconnected, null)); } ZooTrace.logTraceMessage(LOG, ZooTrace.getTextTraceLevel(), "SendThread exited loop for session: 0x" + Long.toHexString(getSessionId())); } |
client 和 server的网络交互
void doTransport(int waitTimeOut, List<Packet> pendingQueue, ClientCnxn cnxn) throws IOException, InterruptedException { try { if (!firstConnect.await(waitTimeOut, TimeUnit.MILLISECONDS)) { return; } Packet head = null; if (needSasl.get()) { if (!waitSasl.tryAcquire(waitTimeOut, TimeUnit.MILLISECONDS)) { return; } } else { //判断outgoingQueue是否存在待发送的数据包,不存在则直接返回 if ((head = outgoingQueue.poll(waitTimeOut, TimeUnit.MILLISECONDS)) == null) { return; } } // check if being waken up on closing. if (!sendThread.getZkState().isAlive()) { // adding back the patck to notify of failure in conLossPacket(). addBack(head); return; } // channel disconnection happened if (disconnected.get()) { //异常流程,channel关闭了,讲当前的packet添加到addBack中 addBack(head); throw new EndOfStreamException("channel for sessionid 0x" + Long.toHexString(sessionId) + " is lost"); } if (head != null) { //如果当前存在需要发送的数据包,则调用doWrite方法,pendingQueue表示处于已经发送过等待响应的packet队列 doWrite(pendingQueue, head, cnxn); } } finally { updateNow(); } } |
DoWrite方法
private void doWrite(List<Packet> pendingQueue, Packet p, ClientCnxn cnxn) { updateNow(); while (true) { if (p != WakeupPacket.getInstance()) { if ((p.requestHeader != null) && //判断请求头以及判断当前请求类型不是ping或者auth操作 (p.requestHeader.getType() != ZooDefs.OpCode.ping) && (p.requestHeader.getType() != ZooDefs.OpCode.auth)) { p.requestHeader.setXid(cnxn.getXid()); //设置xid,这个xid用来区分请求类型 synchronized (pendingQueue) { pendingQueue.add(p); //将当前的packet添加到pendingQueue队列中 } } sendPkt(p); //将数据包发送出去 } if (outgoingQueue.isEmpty()) { break; } p = outgoingQueue.remove(); } } |
sendPkt
private void sendPkt(Packet p) { // Assuming the packet will be sent out successfully. Because if it fails, // the channel will close and clean up queues. p.createBB(); //序列化请求数据 updateLastSend(); //更新最后一次发送updateLastSend sentCount++; //更新发送次数 channel.write(ChannelBuffers.wrappedBuffer(p.bb)); //通过nio channel发送字节缓存到服务端 } |
createBB
public void createBB() { try { ByteArrayOutputStream baos = new ByteArrayOutputStream(); BinaryOutputArchive boa = BinaryOutputArchive.getArchive(baos); boa.writeInt(-1, "len"); // We'll fill this in later if (requestHeader != null) { requestHeader.serialize(boa, "header"); //序列化header头(requestHeader) } if (request instanceof ConnectRequest) { request.serialize(boa, "connect"); // append "am-I-allowed-to-be-readonly" flag boa.writeBool(readOnly, "readOnly"); } else if (request != null) { request.serialize(boa, "request"); //序列化request(request) } baos.close(); this.bb = ByteBuffer.wrap(baos.toByteArray()); this.bb.putInt(this.bb.capacity() - 4); this.bb.rewind(); } catch (IOException e) { LOG.warn("Ignoring unexpected exception", e); } } |
从createBB方法中,我们看到在底层实际的网络传输序列化中,zookeeper只会讲requestHeader和request两个属性进行序列化,即只有这两个会被序列化到底层字节数组中去进行网络传输,不会将watchRegistration相关的信息进行网络传输。
总结
用户调用exists注册监听以后,会做几个事情
- 讲请求数据封装为packet,添加到outgoingQueue
- SendThread这个线程会执行数据发送操作,主要是将outgoingQueue队列中的数据发送到服务端
- 通过clientCnxnSocket.doTransport(to, pendingQueue, ClientCnxn.this); 其中ClientCnxnSocket只zookeeper客户端和服务端的连接通信的封装,有两个具体的实现类ClientCnxnSocketNetty和ClientCnxnSocketNIO;具体使用哪一个类来实现发送,是在初始化过程是在实例化Zookeeper的时候设置的,代码如下
cnxn = new ClientCnxn(connectStringParser.getChrootPath(), hostProvider, sessionTimeout, this, watchManager, getClientCnxnSocket(), canBeReadOnly); |
private ClientCnxnSocket getClientCnxnSocket() throws IOException { String clientCnxnSocketName = getClientConfig().getProperty( ZKClientConfig.ZOOKEEPER_CLIENT_CNXN_SOCKET); if (clientCnxnSocketName == null) { clientCnxnSocketName = ClientCnxnSocketNIO.class.getName(); } try { Constructor<?> clientCxnConstructor = Class.forName(clientCnxnSocketName).getDeclaredConstructor(ZKClientConfig.class); ClientCnxnSocket clientCxnSocket = (ClientCnxnSocket) clientCxnConstructor.newInstance(getClientConfig()); return clientCxnSocket; } catch (Exception e) { IOException ioe = new IOException("Couldn't instantiate " + clientCnxnSocketName); ioe.initCause(e); throw ioe; } } |
4.基于第3步,最终会在ClientCnxnSocketNetty方法中执行sendPkt将请求的数据包发送到服务端
对Java技术,架构技术感兴趣的同学,欢迎加QQ群619881427,一起学习,相互讨论。
群内已经有小伙伴将知识体系整理好(源码,笔记,PPT,学习视频),欢迎加群免费领取。
分享给喜欢Java,喜欢编程,有梦想成为架构师的程序员们,希望能够帮助到你们。
不是Java程序员也没关系,帮忙转发给更多朋友!谢谢。
分享一个小技巧点击阅读原文也可以轻松获取到学习资料哦!!
Zookeeper-watcher机制源码分析(一)的更多相关文章
- Springboot学习04-默认错误页面加载机制源码分析
Springboot学习04-默认错误页面加载机制源码分析 前沿 希望通过本文的学习,对错误页面的加载机制有这更神的理解 正文 1-Springboot错误页面展示 2-Springboot默认错误处 ...
- ApplicationEvent事件机制源码分析
<spring扩展点之三:Spring 的监听事件 ApplicationListener 和 ApplicationEvent 用法,在spring启动后做些事情> <服务网关zu ...
- Android事件分发机制源码分析
Android事件分发机制源码分析 Android事件分发机制源码分析 Part1事件来源以及传递顺序 Activity分发事件源码 PhoneWindow分发事件源码 小结 Part2ViewGro ...
- 浅谈ZooKeeper基本原理与源码分析
最近一直有小伙伴私信我,问一些关于Zookeeper的知识,下边关于的Zookeeper的知识整理了一下,一起学习一下. 看完本文对于Zookeeper想深入全面了解的读者朋友们,小编这里整理了一份更 ...
- 【Cocos2d-x 3.x】 事件处理机制源码分析
在游戏中,触摸是最基本的,必不可少的.Cocos2d-x 3.x中定义了一系列事件,同时也定义了负责监听这些事件的监听器,另外,cocos定义了事件分发类,用来将事件派发出去以便可以实现相应的事件. ...
- Android查缺补漏(View篇)--事件分发机制源码分析
在上一篇博文中分析了事件分发的流程及规则,本篇会从源码的角度更进一步理解事件分发机制的原理,如果对事件分发规则还不太清楚的童鞋,建议先看一下上一篇博文 <Android查缺补漏(View篇)-- ...
- Android异步消息传递机制源码分析
1.Android异步消息传递机制有以下两个方式:(异步消息传递来解决线程通信问题) handler 和 AsyncTask 2.handler官方解释的用途: 1).定时任务:通过handler.p ...
- hadoop的RPC机制 -源码分析
这些天一直奔波于长沙和武汉之间,忙着腾讯的笔试.面试,以至于对hadoop RPC(Remote Procedure Call Protocol ,远程过程调用协议,它是一种通过网络从远程计算机程序上 ...
- 【RabbitMQ学习记录】- 消息队列存储机制源码分析
本文来自 网易云社区 . RabbitMQ在金融系统,OpenStack内部组件通信和通信领域应用广泛,它部署简单,管理界面内容丰富使用十分方便.笔者最近在研究RabbitMQ部署运维和代码架构,本篇 ...
- Hadoop心跳机制源码分析
正文: 一.体系背景 首先和大家说明一下:hadoop的心跳机制的底层是通过RPC机制实现的,这篇文章我只介绍心跳实现的代码,对于底层的具体实现,大家可以参考我的另几篇博客: 1. hadoop的RP ...
随机推荐
- springmvd接收参数问题
问题描述: 好久不写博客了,今天遇到一个问题,那就是post请求时,参数接收不到,当时我很纳闷,看代码: 就是这样几个参数,我使用postman请求时无法获取参数: 报错信息: "msg&q ...
- 合适IT人的健身技巧
合适IT人的健身技巧: 健身益寿生活十条 虽然遗传学家说人的平均寿命可长达120岁,但本世纪人的寿命远远达不到这个数字^有鉴于此,国外一批医生.心理学家和营养学家制定了健身益寿生活10条准则,认为如能 ...
- 如何判断Linux是32位还是64位
getconf LONG_BIT
- mysql编译好的简单安装
.创建mysql用户的账号 useradd mysql .下载mysql编译好的压缩包 .安装mysql yum -y install numactl libaio 安装依赖库 tar -zxvf m ...
- Java容器-个人整理1
1.初始化集合时,若能知道知道容量,尽量初始化时确定容量.容器类一般可以自动扩充,但扩充是有性能代价的. 2.Arrays.asList()的底层表示仍然时数组,因此不能进行调整尺寸的操作. 3.Ha ...
- iOS 关于重定向的那些事(NSURLProcotol-WKWebView)
重定向定义:重定向(Redirect)就是通过各种方法将各种网络请求重新定个方向转到其它位置(如:网页重定向.域名的重定向.路由选择的变化也是对数据报文经由路径的一种重定向). NSURLProt ...
- EL和JSTL笔记
1.EL表达式 [1] 简介 > JSP表达式 <%= %> 用于向页面中输出一个对象. > 到JSP2.0时,在我们的页面中不允许出现 JSP表达式和 脚本片段. > ...
- 2019.03.28 bzoj3594: [Scoi2014]方伯伯的玉米田(二维bit优化dp)
传送门 题意咕咕咕 思路:直接上二维bitbitbit优化dpdpdp即可. 代码: #include<bits/stdc++.h> #define N 10005 #define K 5 ...
- Latex引用\ref
在等式/图/定理等后面加\label{name} 正文引用方法:等式使用 \eqref{name},非等式引用可使用\ref{name}.
- 工程无法正常调试运行unknown failure at android.os.Binder.execTransact
同事正常使用的工程,放到另电脑上,开后可以正常编译,但是无法安装调试到手机上,始终提示错误 新建一个工程正常. 最后通过把开发工具升级到最新版本解决.