问题

  • session 如何生成的?sessionId为什么不直接使用时间戳+单机名
  • sessionid 关闭的时候的逻辑,sessionid 的维护是由各节点还是leader ?

会话相关

sessionid 生成

  我们看一下session 管理类 SessionTrackerImpl。 它主要维护三个字段


  1. //根据 sessionid 存放的 session
  2. HashMap<Long, SessionImpl> sessionsById = new HashMap<Long, SessionImpl>();
  3. //key 是时间 ,用于根据下次会话超时时间点来归纳会话,便于进行会话管理和超时检查,(分桶策略容器)
  4. HashMap<Long, SessionSet> sessionSets = new HashMap<Long, SessionSet>();
  5. //key 是 sessionId , value 是过期时间
  6. ConcurrentHashMap<Long, Integer> sessionsWithTimeout;

  其他的都是操作 session 的方法,我们看一下创建sessionId的过程,SessionImpl 类

  1. public static long initializeNextSession(long id) {
  2. long nextSid = 0;
  3. nextSid = (Time.currentElapsedTime() << 24) >>> 8;
  4. nextSid = nextSid | (id <<56);
  5. return nextSid;
  6. }

  得出的 sessionid前8位确定所在的机器,后56位使用当前时间的毫秒数表示进行随机。

session 管理

  使用的是分桶策略,如下图所示。

  以时间戳为节点,每个桶装这该时间点(过期的时间)的 session集合,然后有一个线程那桶里的多个session 进行检查,加入过期时间被延长,那么session 进行迁移到其他的桶,否则将被清理。

创建会话执行事务过程

  创建会话之前我们要先知道zk服务器是如何和客户端连接的。上一篇文章结尾处,我们知道了处理与 客户端的连接主要是由 NIOServerCnxnFactory 来负责的。而真正处理的逻辑就在 run 方法。

  1. /**
  2. * 默认允许连接 60个客户端,
  3. * 这个(一个)线程会处理
  4. * - 来自客户端的连接
  5. * - 来自客户端的读
  6. * - 来自客户端的写
  7. *
  8. *
  9. */
  10. public void run() {
  11. //只要没有断开,循环一直进行
  12. //下面连接就是我们熟悉的 java NIO 的运用
  13. while (!ss.socket().isClosed()) {
  14. try {
  15. //select 方法一直就阻塞
  16. selector.select(1000);
  17. Set<SelectionKey> selected;
  18. //这里为什么要加锁呢?可能有多个线程
  19. synchronized (this) {
  20. selected = selector.selectedKeys();
  21. }
  22. ArrayList<SelectionKey> selectedList = new ArrayList<SelectionKey>(
  23. selected);
  24. Collections.shuffle(selectedList);
  25. for (SelectionKey k : selectedList) {
  26. if ((k.readyOps() & SelectionKey.OP_ACCEPT) != 0) {
  27. SocketChannel sc = ((ServerSocketChannel) k
  28. .channel()).accept();
  29. InetAddress ia = sc.socket().getInetAddress();
  30. int cnxncount = getClientCnxnCount(ia);
  31. //客户端可以保存 60 个连接
  32. if (maxClientCnxns > 0 && cnxncount >= maxClientCnxns){
  33. LOG.warn("Too many connections from " + ia
  34. + " - max is " + maxClientCnxns );
  35. sc.close();
  36. } else {
  37. LOG.info("Accepted socket connection from "
  38. + sc.socket().getRemoteSocketAddress());
  39. sc.configureBlocking(false);
  40. SelectionKey sk = sc.register(selector,
  41. SelectionKey.OP_READ);
  42. //创建连接,连接用 NIOServerCnxn这个类来维护
  43. NIOServerCnxn cnxn = createConnection(sc, sk);
  44. sk.attach(cnxn);
  45. addCnxn(cnxn);
  46. }
  47. } else if ((k.readyOps() & (SelectionKey.OP_READ | SelectionKey.OP_WRITE)) != 0) {
  48. //处理客户端读写操作,重点看 doIO 方法
  49. NIOServerCnxn c = (NIOServerCnxn) k.attachment();
  50. c.doIO(k);
  51. } else {
  52. if (LOG.isDebugEnabled()) {
  53. LOG.debug("Unexpected ops in select "
  54. + k.readyOps());
  55. }
  56. }
  57. }
  58. selected.clear();
  59. } catch (RuntimeException e) {
  60. LOG.warn("Ignoring unexpected runtime exception", e);
  61. } catch (Exception e) {
  62. LOG.warn("Ignoring exception", e);
  63. }
  64. }
  65. closeAll();
  66. LOG.info("NIOServerCnxn factory exited run method");
  67. }
  1. 我们再来看一下 doIO这个方法 位于 NIOServerCnxn
  1. /**
  2. * Handles read/write IO on connection.
  3. */
  4. void doIO(SelectionKey k) throws InterruptedException {
  5. try {
  6. if (isSocketOpen() == false) {
  7. LOG.warn("trying to do i/o on a null socket for session:0x"
  8. + Long.toHexString(sessionId));
  9. return;
  10. }
  11. //可读类型
  12. if (k.isReadable()) {
  13. int rc = sock.read(incomingBuffer);
  14. if (rc < 0) {
  15. throw new EndOfStreamException(
  16. "Unable to read additional data from client sessionid 0x"
  17. + Long.toHexString(sessionId)
  18. + ", likely client has closed socket");
  19. }
  20. if (incomingBuffer.remaining() == 0) {
  21. boolean isPayload;
  22. //读取下一个请求
  23. if (incomingBuffer == lenBuffer) { // start of next request
  24. //翻转缓存区,可读
  25. incomingBuffer.flip();
  26. //读取lenBuffer的前四个字节,当读取的是内容长度时则为true,否则为false
  27. isPayload = readLength(k);
  28. //清空缓存
  29. incomingBuffer.clear();
  30. } else {
  31. // continuation
  32. isPayload = true;
  33. }
  34. // isPayload 为 true ,表示buffer 里面是负载
  35. if (isPayload) { // not the case for 4letterword
  36. readPayload();
  37. }
  38. else {
  39. // four letter words take care
  40. // need not do anything else
  41. return;
  42. }
  43. }
  44. }
  45. //可写类型
  46. if (k.isWritable()) {
  47. // ZooLog.logTraceMessage(LOG,
  48. // ZooLog.CLIENT_DATA_PACKET_TRACE_MASK
  49. // "outgoingBuffers.size() = " +
  50. // outgoingBuffers.size());
  51. if (outgoingBuffers.size() > 0) {
  52. // ZooLog.logTraceMessage(LOG,
  53. // ZooLog.CLIENT_DATA_PACKET_TRACE_MASK,
  54. // "sk " + k + " is valid: " +
  55. // k.isValid());
  56. /*
  57. * This is going to reset the buffer position to 0 and the
  58. * limit to the size of the buffer, so that we can fill it
  59. * with data from the non-direct buffers that we need to
  60. * send.
  61. */
  62. ByteBuffer directBuffer = factory.directBuffer;
  63. directBuffer.clear();
  64. for (ByteBuffer b : outgoingBuffers) {
  65. if (directBuffer.remaining() < b.remaining()) {
  66. /*
  67. * When we call put later, if the directBuffer is to
  68. * small to hold everything, nothing will be copied,
  69. * so we've got to slice the buffer if it's too big.
  70. */
  71. b = (ByteBuffer) b.slice().limit(
  72. directBuffer.remaining());
  73. }
  74. /*
  75. * put() is going to modify the positions of both
  76. * buffers, put we don't want to change the position of
  77. * the source buffers (we'll do that after the send, if
  78. * needed), so we save and reset the position after the
  79. * copy
  80. */
  81. int p = b.position();
  82. directBuffer.put(b);
  83. b.position(p);
  84. if (directBuffer.remaining() == 0) {
  85. break;
  86. }
  87. }
  88. /*
  89. * Do the flip: limit becomes position, position gets set to
  90. * 0. This sets us up for the write.
  91. */
  92. directBuffer.flip();
  93. int sent = sock.write(directBuffer);
  94. ByteBuffer bb;
  95. // Remove the buffers that we have sent
  96. while (outgoingBuffers.size() > 0) {
  97. bb = outgoingBuffers.peek();
  98. if (bb == ServerCnxnFactory.closeConn) {
  99. throw new CloseRequestException("close requested");
  100. }
  101. int left = bb.remaining() - sent;
  102. if (left > 0) {
  103. /*
  104. * We only partially sent this buffer, so we update
  105. * the position and exit the loop.
  106. */
  107. bb.position(bb.position() + sent);
  108. break;
  109. }
  110. packetSent();
  111. /* We've sent the whole buffer, so drop the buffer */
  112. sent -= bb.remaining();
  113. outgoingBuffers.remove();
  114. }
  115. // ZooLog.logTraceMessage(LOG,
  116. // ZooLog.CLIENT_DATA_PACKET_TRACE_MASK, "after send,
  117. // outgoingBuffers.size() = " + outgoingBuffers.size());
  118. }
  119. synchronized(this.factory){
  120. if (outgoingBuffers.size() == 0) {
  121. if (!initialized
  122. && (sk.interestOps() & SelectionKey.OP_READ) == 0) {
  123. throw new CloseRequestException("responded to info probe");
  124. }
  125. sk.interestOps(sk.interestOps()
  126. & (~SelectionKey.OP_WRITE));
  127. } else {
  128. sk.interestOps(sk.interestOps()
  129. | SelectionKey.OP_WRITE);
  130. }
  131. }
  132. }
  133. } catch (CancelledKeyException e) {
  134. LOG.warn("CancelledKeyException causing close of session 0x"
  135. + Long.toHexString(sessionId));
  136. if (LOG.isDebugEnabled()) {
  137. LOG.debug("CancelledKeyException stack trace", e);
  138. }
  139. close();
  140. } catch (CloseRequestException e) {
  141. // expecting close to log session closure
  142. close();
  143. } catch (EndOfStreamException e) {
  144. LOG.warn(e.getMessage());
  145. if (LOG.isDebugEnabled()) {
  146. LOG.debug("EndOfStreamException stack trace", e);
  147. }
  148. // expecting close to log session closure
  149. close();
  150. } catch (IOException e) {
  151. LOG.warn("Exception causing close of session 0x"
  152. + Long.toHexString(sessionId) + ": " + e.getMessage());
  153. if (LOG.isDebugEnabled()) {
  154. LOG.debug("IOException stack trace", e);
  155. }
  156. close();
  157. }
  158. }
  159. /** Read the request payload (everything following the length prefix) */
  160. private void readPayload() throws IOException, InterruptedException {
  161. if (incomingBuffer.remaining() != 0) { // have we read length bytes?
  162. int rc = sock.read(incomingBuffer); // sock is non-blocking, so ok
  163. if (rc < 0) {
  164. throw new EndOfStreamException(
  165. "Unable to read additional data from client sessionid 0x"
  166. + Long.toHexString(sessionId)
  167. + ", likely client has closed socket");
  168. }
  169. }
  170. if (incomingBuffer.remaining() == 0) { // have we read length bytes?
  171. packetReceived();
  172. incomingBuffer.flip();
  173. //执行读取负载的逻辑
  174. if (!initialized) {
  175. //非初始化
  176. readConnectRequest();
  177. } else {
  178. readRequest();
  179. }
  180. lenBuffer.clear();
  181. incomingBuffer = lenBuffer;
  182. }
  183. }
  184. 其中我们看一下如何连接的,即上面的 readConnectRequest 方法
  185. private void readConnectRequest() throws IOException, InterruptedException {
  186. if (!isZKServerRunning()) {
  187. throw new IOException("ZooKeeperServer not running");
  188. }
  189. zkServer.processConnectRequest(this, incomingBuffer);
  190. initialized = true;
  191. }
  1. 于是我们到了ZookeeperServer processConnectRequest 方法
  1. /**
  2. * 封装成一个 connectRequest
  3. * 提交请求到leader
  4. */
  5. public void processConnectRequest(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
  6. BinaryInputArchive bia = BinaryInputArchive.getArchive(new ByteBufferInputStream(incomingBuffer));
  7. ConnectRequest connReq = new ConnectRequest();
  8. connReq.deserialize(bia, "connect");
  9. if (LOG.isDebugEnabled()) {
  10. LOG.debug("Session establishment request from client "
  11. + cnxn.getRemoteSocketAddress()
  12. + " client's lastZxid is 0x"
  13. + Long.toHexString(connReq.getLastZxidSeen()));
  14. }
  15. boolean readOnly = false;
  16. try {
  17. readOnly = bia.readBool("readOnly");
  18. cnxn.isOldClient = false;
  19. } catch (IOException e) {
  20. // this is ok -- just a packet from an old client which
  21. // doesn't contain readOnly field
  22. LOG.warn("Connection request from old client "
  23. + cnxn.getRemoteSocketAddress()
  24. + "; will be dropped if server is in r-o mode");
  25. }
  26. if (readOnly == false && this instanceof ReadOnlyZooKeeperServer) {
  27. String msg = "Refusing session request for not-read-only client "
  28. + cnxn.getRemoteSocketAddress();
  29. LOG.info(msg);
  30. throw new CloseRequestException(msg);
  31. }
  32. if (connReq.getLastZxidSeen() > zkDb.dataTree.lastProcessedZxid) {
  33. String msg = "Refusing session request for client "
  34. + cnxn.getRemoteSocketAddress()
  35. + " as it has seen zxid 0x"
  36. + Long.toHexString(connReq.getLastZxidSeen())
  37. + " our last zxid is 0x"
  38. + Long.toHexString(getZKDatabase().getDataTreeLastProcessedZxid())
  39. + " client must try another server";
  40. LOG.info(msg);
  41. throw new CloseRequestException(msg);
  42. }
  43. int sessionTimeout = connReq.getTimeOut();
  44. byte passwd[] = connReq.getPasswd();
  45. int minSessionTimeout = getMinSessionTimeout();
  46. if (sessionTimeout < minSessionTimeout) {
  47. sessionTimeout = minSessionTimeout;
  48. }
  49. int maxSessionTimeout = getMaxSessionTimeout();
  50. if (sessionTimeout > maxSessionTimeout) {
  51. sessionTimeout = maxSessionTimeout;
  52. }
  53. cnxn.setSessionTimeout(sessionTimeout);
  54. // We don't want to receive any packets until we are sure that the
  55. // session is setup
  56. cnxn.disableRecv();
  57. long sessionId = connReq.getSessionId();
  58. if (sessionId != 0) {
  59. long clientSessionId = connReq.getSessionId();
  60. LOG.info("Client attempting to renew session 0x"
  61. + Long.toHexString(clientSessionId)
  62. + " at " + cnxn.getRemoteSocketAddress());
  63. serverCnxnFactory.closeSession(sessionId);
  64. cnxn.setSessionId(sessionId);
  65. reopenSession(cnxn, sessionId, passwd, sessionTimeout);
  66. } else {
  67. LOG.info("Client attempting to establish new session at "
  68. + cnxn.getRemoteSocketAddress());
  69. //真正执行请求的地方 : 创建 session,提交请求
  70. createSession(cnxn, passwd, sessionTimeout);
  71. }
  72. }
  73. long createSession(ServerCnxn cnxn, byte passwd[], int timeout) {
  74. long sessionId = sessionTracker.createSession(timeout);
  75. Random r = new Random(sessionId ^ superSecret);
  76. r.nextBytes(passwd);
  77. ByteBuffer to = ByteBuffer.allocate(4);
  78. to.putInt(timeout);
  79. cnxn.setSessionId(sessionId);
  80. //提交请求
  81. submitRequest(cnxn, sessionId, OpCode.createSession, 0, to, null);
  82. return sessionId;
  83. }
  84. /**
  85. * @param cnxn
  86. * @param sessionId
  87. * @param xid
  88. * @param bb
  89. */
  90. private void submitRequest(ServerCnxn cnxn, long sessionId, int type,
  91. int xid, ByteBuffer bb, List<Id> authInfo) {
  92. Request si = new Request(cnxn, sessionId, xid, type, bb, authInfo);
  93. submitRequest(si);
  94. }
  95. public void submitRequest(Request si) {
  96. if (firstProcessor == null) {
  97. synchronized (this) {
  98. try {
  99. // Since all requests are passed to the request
  100. // processor it should wait for setting up the request
  101. // processor chain. The state will be updated to RUNNING
  102. // after the setup.
  103. // 在 startup 方法中直到初始化完成才得以接受请求
  104. while (state == State.INITIAL) {
  105. wait(1000);
  106. }
  107. } catch (InterruptedException e) {
  108. LOG.warn("Unexpected interruption", e);
  109. }
  110. if (firstProcessor == null || state != State.RUNNING) {
  111. throw new RuntimeException("Not started");
  112. }
  113. }
  114. }
  115. try {
  116. //判断 session 是否还存活
  117. touch(si.cnxn);
  118. boolean validpacket = Request.isValid(si.type);
  119. if (validpacket) {
  120. //firstProcessor 开始执行
  121. firstProcessor.processRequest(si);
  122. if (si.cnxn != null) {
  123. incInProcess();
  124. }
  125. } else {
  126. LOG.warn("Received packet at server of unknown type " + si.type);
  127. new UnimplementedRequestProcessor().processRequest(si);
  128. }
  129. } catch (MissingSessionException e) {
  130. if (LOG.isDebugEnabled()) {
  131. LOG.debug("Dropping request: " + e.getMessage());
  132. }
  133. } catch (RequestProcessorException e) {
  134. LOG.error("Unable to process request:" + e.getMessage(), e);
  135. }
  136. }

  清楚了服务器与客户端的连接,知道了最终会到执行任务链上。

  我们可以连接任意一台 zk服务器,进行事务请求,当follower接受到请求后它就会转发给leader ,而follower 和 leader 都是使用责任链进行处理来自客户端的请求的。

  我们先来看一下follower 和 leader 责任链的流程。



请求处理流程

会话创建请求

  我们通过下面的图片先来了解整个过程



  概括起来就是 :

  • NIOServerCnxn 接受请求
  • 协商 sessionTimeout ,创建 connectRequest
  • 创建会话,生成 sessionId,注册会话和激活会话
  • 交给 leader 的 PrepRequestProcessor
  • 创建请求事务体 createSessionTxn
  • 交给 ProposalRequestProcessor,接下来就会进入三个子流程

      子流程如下 :
  • Sync流程,follower做好日志记录,同时返回 ACK 给leader
  • Proposal流程,生成proposal ,广播提议,获得半数票后,请求加入到 toBeApplied 队列,广播commit 信息
  • Commit流程,将请求交付给CommitProcessor 处理器,等待上阶段的投票结果,提交请求,交付给下一个处理器 : FinalRequestProcessor
  • 到此三个子流程走完后,到了最后的阶段,事务应用,之前我们的议案只是应用在日志中,并没有在内存中生效,这阶段需要将请求应用在内存中。

      这个就是整个会话创建请求的处理过程,当客户端发出事务请求时也是像处理会话创建一样的流程。

总结

  这一篇主要讲了zk 创建session的策略和管理的逻辑,同时介绍了处理事务的过程。

zookeeper 源码(二) session 和 处理事务请求的更多相关文章

  1. zookeeper源码 — 二、集群启动—leader选举

    上一篇介绍了zookeeper的单机启动,集群模式下启动和单机启动有相似的地方,但是也有各自的特点.集群模式的配置方式和单机模式也是不一样的,这一篇主要包含以下内容: 概念介绍:角色,服务器状态 服务 ...

  2. ZooKeeper源码阅读——client(二)

    原创技术文章,转载请注明:转自http://newliferen.github.io/ 如何连接ZooKeeper集群   要想了解ZooKeeper客户端实现原理,首先需要关注一下客户端的使用方式, ...

  3. Zookeeper 源码(二)序列化组件 Jute

    Zookeeper 源码(二)序列化组件 Jute 一.序列化组件 Jute 对于一个网络通信,首先需要解决的就是对数据的序列化和反序列化处理,在 ZooKeeper 中,使用了Jute 这一序列化组 ...

  4. zookeeper源码分析之五服务端(集群leader)处理请求流程

    leader的实现类为LeaderZooKeeperServer,它间接继承自标准ZookeeperServer.它规定了请求到达leader时需要经历的路径: PrepRequestProcesso ...

  5. zookeeper源码分析之四服务端(单机)处理请求流程

    上文: zookeeper源码分析之一服务端启动过程 中,我们介绍了zookeeper服务器的启动过程,其中单机是ZookeeperServer启动,集群使用QuorumPeer启动,那么这次我们分析 ...

  6. Zookeeper 源码(七)请求处理

    Zookeeper 源码(七)请求处理 以单机启动为例讲解 Zookeeper 是如何处理请求的.先回顾一下单机时的请求处理链. // 单机包含 3 个请求链:PrepRequestProcessor ...

  7. Zookeeper 源码(六)Leader-Follower-Observer

    Zookeeper 源码(六)Leader-Follower-Observer 上一节介绍了 Leader 选举的全过程,本节讲解一下 Leader-Follower-Observer 服务器的三种角 ...

  8. Zookeeper 源码(五)Leader 选举

    Zookeeper 源码(五)Leader 选举 前面学习了 Zookeeper 服务端的相关细节,其中对于集群启动而言,很重要的一部分就是 Leader 选举,接着就开始深入学习 Leader 选举 ...

  9. Zookeeper 源码(四)Zookeeper 服务端源码

    Zookeeper 源码(四)Zookeeper 服务端源码 Zookeeper 服务端的启动入口为 QuorumPeerMain public static void main(String[] a ...

  10. Zookeeper 源码(三)Zookeeper 客户端源码

    Zookeeper 源码(三)Zookeeper 客户端源码 Zookeeper 客户端主要有以下几个重要的组件.客户端会话创建可以分为三个阶段:一是初始化阶段.二是会话创建阶段.三是响应处理阶段. ...

随机推荐

  1. cadence动态铜皮的参数设置

    注意这幅图和上一幅图那个焊盘距离shape的间距,变大了,这个设置很用的.

  2. 从原理到方案,一步步讲解web移动端实现自适应等比缩放

    前言 在移动端做自适应,我们常用的有媒体查询,rem ,em,宽度百分比这几种方案.但是都各有其缺点. 首先拿媒体查询来说,在某一个宽度区间内只能使用一种样式,为了适应不同屏幕要,css的代码量就会增 ...

  3. uevent机制

    uevent, user space event. 内核与用户空间的一种通信机制,基于netlink机制,主要用于设备驱动模型,例如热插拔. 1.调用/sbin/mdev的流程分析 在驱动程序中经常出 ...

  4. mvc 模板位置

    mvc4 模板位置 Common7\IDE\ItemTemplates\CSharp\Web\MVC 4\CodeTemplates mvc5 模板位置 Common7\IDE\Extensions\ ...

  5. CentOS7下使用C/C++连接MariaDB/MySQL

    前言 连接数据库通常在Java中使用比较多,但是C/C++在Linux下操作数据库也是比较重要的,很多时候都能用得到,在网上查了很多教程,大多写的有些问题,通过自己摸索,终于成功的连接了MariaDB ...

  6. blur()低通滤波

    blur()函数可以用标准化的盒式过滤器来平滑图像. C++ API: 相关官网资料: https://docs.opencv.org/3.4.1/d4/d86/group__imgproc__fil ...

  7. Java 读取网络资源文件 获取文件大小 MD5校验值

    Java 读取网络资源文件 获取文件大小 MD5校验值 封装一个文件操作工具类: package c; import java.io.*; import java.net.HttpURLConnect ...

  8. angularJS 十六进制与字符串相互转换

    angular 将字符串数据转换为十六进制数据 /** * @Description: TODO 字符串转16进制方法 * @author wjw * @date 2019年9月18日16:35:32 ...

  9. windows系统 安装 mysql.fx

    windows系统 安装 mqtt.fx 软件官网:http://mqttfx.jfx4ee.org/ 软件下载:http://www.jensd.de/apps/mqttfx/1.1.0/

  10. IDEA格式化代码快捷键失灵原因

    网易云音乐快捷键与IDEA快捷键冲突了!!!!!!!坑爹