TCP套接字选项SO_LINGER与TCP_LINGER2

概述

本文对两个LINGER相关的套接字选项进行源码层面的分析，以更明确其各自的作用和区别；

man page

SO_LINGER，该选项是socket层面的选项，通过struct linger结构来设置信息，如果启用该选项，那么使用close()和shutdown()(注意：虽然manpage这么写，但是shutdown内核代码流程中并未用到该选项)关闭socket，将会等待发送队列中的数据发送完成或者等待超时；如果不启用该选项，那么调用会立即返回，关闭任务在后台完成；注意：如果是调用exit()函数关闭socket，那么无论是否启用SO_LINGER选项，socket总会在后台执行linger等待；

        SO_LINGER
               Sets or gets the SO_LINGER option.  The argument is a linger
               structure.

                   struct linger {
                       int l_onoff;    /* linger active */
                       int l_linger;   /* how many seconds to linger for */
                   };

               When enabled, a close() or shutdown() will not return until
               all queued messages for the socket have been successfully sent
               or the linger timeout has been reached.  Otherwise, the call
               returns immediately and the closing is done in the background.
               When the socket is closed as part of exit(), it always
               lingers in the background.

TCP_LINGER2，该选项是TCP层面的，用于设定孤儿套接字在FIN_WAIT2状态的生存时间，该选项可以用来替代系统级别的tcp_fin_timeout配置；在用于移植的代码中不应该使用该选项；另外，需要注意，不要混淆该选项与socket的SO_LINGER选项；

        TCP_LINGER2 (since Linux 2.4)
               The lifetime of orphaned FIN_WAIT2 state sockets.  This option
               can be used to override the system-wide setting in the file
               /proc/sys/net/ipv4/tcp_fin_timeout for this socket.  This is
               not to be confused with the socket() level option SO_LINGER.
               This option should not be used in code intended to be
               portable.

源码分析

SO_LINGER

在调用close()系统调用时，如果引用计数已经为0，则会进行套接字关闭操作，我们从inet_release开始分析；前置步骤请移步<套接字之close系统调用>；如果启用了SO_LINGER选项，那么会将lingertime传入到传输层的关闭函数中，tcp为tcp_close；

 /*
  *    The peer socket should always be NULL (or else). When we call this
  *    function we are destroying the object and from then on nobody
  *    should refer to it.
  */
 int inet_release(struct socket *sock)
 {
     struct sock *sk = sock->sk;

     if (sk) {
         long timeout;

         /* Applications forget to leave groups before exiting */
         /* 退出组播组 */
         ip_mc_drop_socket(sk);

         /* If linger is set, we don't return until the close
          * is complete.  Otherwise we return immediately. The
          * actually closing is done the same either way.
          *
          * If the close is due to the process exiting, we never
          * linger..
          */
         timeout = ;

         /*
             设置了linger标记，进程未在退出，
             则设置lingertime延迟关闭时间
         */
         if (sock_flag(sk, SOCK_LINGER) &&
             !(current->flags & PF_EXITING))
             timeout = sk->sk_lingertime;
         sock->sk = NULL;

         /* 调用传输层的close函数 */
         sk->sk_prot->close(sk, timeout);
     }
     return ;
 }

tcp_close函数，在关闭socket销毁资源之前，调用sk_stream_wait_close函数等待数据发送完毕或者达到lingertime超时时间，然后才继续进入关闭socket销毁资源的流程；

 void tcp_close(struct sock *sk, long timeout)
 {
        /* ... */

     /* If socket has been already reset (e.g. in tcp_reset()) - kill it. */
     /* CLOSE状态 */
     if (sk->sk_state == TCP_CLOSE)
         goto adjudge_to_death;

     /* As outlined in RFC 2525, section 2.17, we send a RST here because
      * data was lost. To witness the awful effects of the old behavior of
      * always doing a FIN, run an older 2.1.x kernel or 2.0.x, start a bulk
      * GET in an FTP client, suspend the process, wait for the client to
      * advertise a zero window, then kill -9 the FTP client, wheee...
      * Note: timeout is always zero in such a case.
      */
     /* 修复状态，断开连接 */
     if (unlikely(tcp_sk(sk)->repair)) {
         sk->sk_prot->disconnect(sk, );
     }
     /* 用户进程有数据未读 */
     else if (data_was_unread) {
         /* Unread data was tossed, zap the connection. */
         NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONCLOSE);

         /* 设置为close */
         tcp_set_state(sk, TCP_CLOSE);

         /* 发送rst */
         tcp_send_active_reset(sk, sk->sk_allocation);
     }
     /* lingertime==0，断开连接 */
     else if (sock_flag(sk, SOCK_LINGER) && !sk->sk_lingertime) {
         /* Check zero linger _after_ checking for unread data. */
         sk->sk_prot->disconnect(sk, );
         NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPABORTONDATA);
     }
     /* 关闭状态转移 */
     else if (tcp_close_state(sk)) {
         /* We FIN if the application ate all the data before
          * zapping the connection.
          */

         /* RED-PEN. Formally speaking, we have broken TCP state
          * machine. State transitions:
          *
          * TCP_ESTABLISHED -> TCP_FIN_WAIT1
          * TCP_SYN_RECV    -> TCP_FIN_WAIT1 (forget it, it's impossible)
          * TCP_CLOSE_WAIT -> TCP_LAST_ACK
          *
          * are legal only when FIN has been sent (i.e. in window),
          * rather than queued out of window. Purists blame.
          *
          * F.e. "RFC state" is ESTABLISHED,
          * if Linux state is FIN-WAIT-1, but FIN is still not sent.
          *
          * The visible declinations are that sometimes
          * we enter time-wait state, when it is not required really
          * (harmless), do not send active resets, when they are
          * required by specs (TCP_ESTABLISHED, TCP_CLOSE_WAIT, when
          * they look as CLOSING or LAST_ACK for Linux)
          * Probably, I missed some more holelets.
          *                         --ANK
          * XXX (TFO) - To start off we don't support SYN+ACK+FIN
          * in a single packet! (May consider it later but will
          * probably need API support or TCP_CORK SYN-ACK until
          * data is written and socket is closed.)
          */
         /* 发送fin */
         tcp_send_fin(sk);
     }

     /*  等待关闭，无数据发送或sk_lingertime超时 */
     sk_stream_wait_close(sk, timeout);

 adjudge_to_death:
     /* socket关闭，释放资源 */
         /* ... */
 }

下面的sk_stream_closing函数检查连接状态，当为TCPF_FIN_WAIT1 | TCPF_CLOSING | TCPF_LAST_ACK时，说明还有数据要发送，这时返回1，等待继续执行；sk_stream_wait_close在等待连接状态不为上述状态时，或者有信号要处理，或者超过lingertime，则返回；

 /**
  * sk_stream_closing - Return 1 if we still have things to send in our buffers.
  * @sk: socket to verify
  */
 static inline int sk_stream_closing(struct sock *sk)
 {
     return ( << sk->sk_state) &
            (TCPF_FIN_WAIT1 | TCPF_CLOSING | TCPF_LAST_ACK);
 }

 void sk_stream_wait_close(struct sock *sk, long timeout)
 {
     if (timeout) {
         DEFINE_WAIT_FUNC(wait, woken_wake_function);

         add_wait_queue(sk_sleep(sk), &wait);

         do {
             if (sk_wait_event(sk, &timeout, !sk_stream_closing(sk), &wait))
                 break;
         } while (!signal_pending(current) && timeout);

         remove_wait_queue(sk_sleep(sk), &wait);
     }
 }

TCP_LINGER2

启动FIN_WAIT_2定时器两个相关逻辑差不多，所以只拿一个位置来说明；在tcp_close函数中，如果判断状态为FIN_WAIT2，则需要进一步判断linger2配置；如下所示，在linger2<0的情况下，关闭连接到CLOSE状态，并且发送rst；在linger2 >= 0的情况下，需判断该值与TIME_WAIT等待时间TCP_TIMEWAIT_LEN值的关系，如果linger2 > TCP_TIMEWAIT_LEN，则启动FIN_WAIT_2定时器，其超时时间为二者的差值；如果linger2<0，则直接进入到TIME_WAIT状态，该TIME_WAIT的子状态是FIN_WAIT2，实际上就是由TIME_WAIT控制块进行了接管，统一交给TIME_WAIT控制块来处理；详细处理过程，后续补充；

 void tcp_close(struct sock *sk, long timeout)
 {
         /* ... */
     if (sk->sk_state == TCP_FIN_WAIT2) {
         struct tcp_sock *tp = tcp_sk(sk);
         /* linger2小于0，无需等待 */
         if (tp->linger2 < ) {

             /* 转到CLOSE */
             tcp_set_state(sk, TCP_CLOSE);
             /* 发送rst */
             tcp_send_active_reset(sk, GFP_ATOMIC);
             __NET_INC_STATS(sock_net(sk),
                     LINUX_MIB_TCPABORTONLINGER);
         } else {

             /* 获取FIN_WAIT_2超时时间 */
             const int tmo = tcp_fin_time(sk);

             /* FIN_WAIT_2超时时间> TIME_WAIT时间，加FIN_WAIT_2定时器 */
             if (tmo > TCP_TIMEWAIT_LEN) {
                 inet_csk_reset_keepalive_timer(sk,
                         tmo - TCP_TIMEWAIT_LEN);
             }
             /* 小于TIME_WAIT时间，则进入TIME_WAIT */
             else {
                 tcp_time_wait(sk, TCP_FIN_WAIT2, tmo);
                 goto out;
             }
         }
     }

         /* ... */
 }

tcp_fin_time函数用来获取通过选项配置的linger2时间，未配置则默认为系统级别的tcp_fin_timeout；

 static inline int tcp_fin_time(const struct sock *sk)
 {
     int fin_timeout = tcp_sk(sk)->linger2 ? : sock_net(sk)->ipv4.sysctl_tcp_fin_timeout;
     const int rto = inet_csk(sk)->icsk_rto;

     if (fin_timeout < (rto << ) - (rto >> ))
         fin_timeout = (rto << ) - (rto >> );

     return fin_timeout;
 }