内存管理pbuf.c源码解析——LwIP学习

声明：个人所写所有博客均为自己在学习中的记录与感想，或为在学习中总结他人学习成果，但因本人才疏学浅，如果大家在阅读过程中发现错误，欢迎大家指正。

本文自己尚有认为写的不完整的地方，源代码没有完全理清，以后会不定期更新

　　上一篇分析了pbuf.h头文件，这次来分析LwIP的内核(core文件夹)中的pbuf.c源代码。本人使用的LwIP源代码为Lwip-1.4.1版本。
　　pbuf.h文件分析地址：http://blog.csdn.net/angel_94/article/details/50111163

一、关于pbuf.c

　　LwIP的内核(core文件夹)文件中pbuf.c是包含协议栈内核使用的数据包管理函数，用于协议栈层次间的数据传递，避免数据拷贝。我们来分析pbuf.c源代码。

其中主要包含的函数：

　　pbuf_alloc()：内存申请函数
　　pbuf_realloc()：调整收缩pbuf的大小，在相应pbuf(链表)尾部释放一定的空间，将数据包pbuf中的数据长度减少为某个长度值
　　pbuf_header()：调整payload指针和长度字段以便为pbuf中的数据预置包头，常用于实现对pbuf预留孔间的操作
　　pbuf_free()：数据包释放函数
　　pbuf_ref()：用于将pbuf中的ref加1
　　pbuf_chain()：用于连接pbufs，连接两个pbuf(链表)为一个pbuf链表
　　pbuf_dechain()：用于连接pbufs
　　pbuf_copy()：用于将一个任何类型的pbuf中的数据拷贝到一个PBUf_RAM类型的pbuf中
　　pbuf_take()：用于向pbuf的数据区域拷贝数据

二、内存申请函数 pbuf_alloc()

1、关于pbuf

　　内存申请函数是这个文件中最重要的函数
　　pbuf是LwIP信息包的内部表示，为最小限度协议栈的特殊需求而设计。pbufs与BSD实现中使用的mbufs相似。pbuf结构即支持动态内存分配保存信息包内容，也支持让信息包数据驻留在静态存储区。pbufs可以在一个链表中链接在一起，被称为一个pbuf链，这样一个信息包可以穿越几个pbufs。
　　pbufs有四种类型：PBUF_RAM、PBUF_ROM、PBUF_REF、PBUF_POOL。
　　程序经常使用LWIP_DEBUGF()函数：LWIP_DEBUGF()是LwIP协议栈的调试信息输出函数
　　1.可以查看函数的调用关系，跟踪程序流程
　　2.查看各种协议的调试信息，关键变量的值

2、pbuf_alloc()函数

/**
 * Allocates a pbuf of the given type (possibly a chain for PBUF_POOL type).
 *分配一个给定类型(可能是一个PBUF_POOL型的链)的pbuf
 *
 * The actual memory allocated for the pbuf is determined by the
 * layer at which the pbuf is allocated and the requested size
 * (from the size parameter).
 *实际内存分配的pbuf是由分配的pbuf的层次和请求的大小决定的
 *
 * @param layer flag to define header size
 * @param length size of the pbuf's payload
 * @param type this parameter decides how and where the pbuf
 * should be allocated as follows:
 *
 * - PBUF_RAM: buffer memory for pbuf is allocated as one large
 *             chunk. This includes protocol headers as well.
 * PBUF_RAM：为pbuf缓冲存储器分配一大块。这也包括协议头
 *
 * - PBUF_ROM: no buffer memory is allocated for the pbuf, even for
 *             protocol headers. Additional headers must be prepended
 *             by allocating another pbuf and chain in to the front of
 *             the ROM pbuf. It is assumed that the memory used is really
 *             similar to ROM in that it is immutable and will not be
 *             changed. Memory which is dynamic should generally not
 *没有缓冲内存分配给PBUF，即使是协议头。附加协议头必须预先在到ROM中的pbuf之前分配另一个pbuf和链表考虑
 *假定内存使用的内存和ROM是非常相似的，它是不可改变的而且不会改变。
 *内存是动态的，一般不应附着到PBUF_ROM中的pbuf。使用PBUF_REF相反。
 *
 * - PBUF_REF: no buffer memory is allocated for the pbuf, even for
 *             protocol headers. It is assumed that the pbuf is only
 *             being used in a single thread. If the pbuf gets queued,
 *             then pbuf_take should be called to copy the buffer.
 *没有缓冲内存分配给PBUF，即使是协议头。假定这个pbuf只被使用在单个线程中。
 *如果pbuf被排队，然后pbuf_take应该叫做复制缓冲区
 *
 * - PBUF_POOL: the pbuf is allocated as a pbuf chain, with pbufs from
 *              the pbuf pool that is allocated during pbuf_init().
 *PBUF_POOL：所述的pbuf被分配为pbuf链，与来自pbuf池的pbufs在pbuf_init()即内存初始化函数中被分配
 *
 *
 * @return the allocated pbuf. If multiple pbufs where allocated, this
 * is the first pbuf of a pbuf chain.
 * 返回被分配的pbuf。如果多个pbuf被分配，这是pbuf链中的第一个pbuf
 */
struct pbuf *
pbuf_alloc(pbuf_layer layer, u16_t length, pbuf_type type)
{
    struct pbuf *p,*q, *r;
    u16_t offset; /*有效数据起始偏移位置*/
    s32_t rem_len; /* remaining length 剩余长度*/
    LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE,("pbuf_alloc(length = %"U16_F")\n", length);

/* determine header offset */
seitch(layer) { /* pbuf的层次 */
case PBUF_TRANSPORT:
    /* add room for transport (often TCP) layer header */
    /* 如果为传输层申请pbuf，那么有效数据的偏移位置为PBUF_LINK_HLEN +PBUF_IP_HLEN +PBUF_TRANSPPORT_HLEN */
    offset  = PBUF_LINK_HLEN + PBUF_IP_HLEN + PBUF_TRANSPPORT_HLEN;
    break；
case PBUF_IP:
    /* add room for IP layer header */
    /* 如果为ip层申请pbuf，那么有效数据的偏移位置为PBUF_ip_HLEN + PBUF_LINK_HLEN */
    offset= PBUF_LINK_HLEN + PBUF_IP_HLEN;
case PBUF_LINK:
    /* add room for link layer header */
    /* 如果是链路层申请pbuf内存，那么有效数据的偏移位置为PBUF_LINK_HLEN */
    offset = PBUF_LINK_HLEN;
    break;
case PBUF_RAM:
    /*如果为原始层申请pbuf内存，那么数据偏移位置就是0，不预留任何空间*/
    offset= 0；
    break;
default：
    LWIP_ASSERT("pbuf_alloc:bad pbuf layer", 0);
    return NULL;
    }

 /********************************************************/
switch (type){ /* pbuf的类型 */
    case PBUF_POOL: /*PBUF_POOL分配一个链表，链表上每个元素所管理的内存最大不超过PBUF_POOL_BUFSIZE*/
        /* allocate head of pbuf chain into p */
        p = (struct pbuf *)memep_malloc(MEMP_PBUF_POOL);/* 为PBUF_POOL类型时 */
        /*PBUF_POOL类型和PBUF_RAM类型的都由内存池分配得到，此处调用了memp_malloc，在memp.c文件中 */
        /* 分配的内存池类型为MEMP_PBUF_POOL*/
        LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_alloc: allocated pbuf %p\n"), (void *)p)); /* p是pbuf类型的指针*/
        /*LWIP_DEBUGF是LwIP协议栈的调试信息输出函数 */
        /*1.可以查看函数的调用关系，跟踪程序流程。*/
        /*2.查看各种协议的调试信息，关键变量的值*/
        if(p == NULL){
            PBUF_POOL_IS_EMPTY();
            return NULL;
        }
        p->type = type;
        p->next = NULL;
        /* make the payload pointer point 'offset' bytes into pbuf data memory */
        p->payload = LWIP_MEM_ALIGN((void *)((u8_t *)p + (SIZEOF_STRUCT_PBUF + offset)));
        /* payload指向pbuf管理的数据的起始地址，payload指向的起始地址即为offset。*/
        /*以offset为基准，登记有效数据存储的起始偏移位置到p->payload[luther.gliethttp] */
        LWIP_ASSERT("pbuf_alloc: pbuf p->payload properly aligned",
            ((mem_ptr_t)p->payload % MEM_ALIGNMENT) == 0);
    /* the total length of the pbuf chain is the requested size */
    p->tot_len = length; /* pbuf链表上有效数据总大小 */
    /* set the length of the first pbuf in the chain */
    p->len = LWIP_MIN(length, PBUF_POOL_BUFSIZE_ALIGNED - LWIP_MEM_ALIGN_SIZE(offset));
    /*pbuf链表上的每个元素所能存储的最大数据为PBUF_POOL_BUFSIZE。如果超过，如果超过该值，那么就会使用链表方式。*/
    /*链接其他很多个pbuf，直到申请的size数据全部能够正常存储为止[luther.gliethttp]*/

    LWIP_ASSERT("check p->payload + p->len does not overflow pbuf",
                ((u8_t*)p->payload + p->len <=
                 (u8_t*)p + SIZEOF_STRUCT_PBUF + PBUF_POOL_BUFSIZE_ALIGNED));
    LWIP_ASSERT("PBUF_POOL_BUFSIZE must be bigger than MEM_ALIGNMENT",
      (PBUF_POOL_BUFSIZE_ALIGNED - LWIP_MEM_ALIGN_SIZE(offset)) > 0 );
    /* set reference count (needed here in case we fail) */
    p->ref = 1; /* ref表示当前pbuf被引用的次数 */

    /* now allocate the tail of the pbuf chain */

    /* remember first pbuf for linkage in next iteration */
    r = p;
    /* remaining length to be allocated */
    rem_len = length - p->len;
    /* any remaining pbufs to be allocated? */
    while (rem_len > 0) {
      q = (struct pbuf *)memp_malloc(MEMP_PBUF_POOL);
      if (q == NULL) {
        PBUF_POOL_IS_EMPTY();
        /* free chain so far allocated */
        pbuf_free(p);
        /* bail out unsuccesfully */
        return NULL;
      }
      q->type = type;
      q->flags = 0;
      q->next = NULL;
      /* make previous pbuf point to this pbuf */
      r->next = q;
      /* set total length of this pbuf and next in chain */
      LWIP_ASSERT("rem_len < max_u16_t", rem_len < 0xffff);
      q->tot_len = (u16_t)rem_len;
      /* this pbuf length is pool size, unless smaller sized tail */
      q->len = LWIP_MIN((u16_t)rem_len, PBUF_POOL_BUFSIZE_ALIGNED);
      q->payload = (void *)((u8_t *)q + SIZEOF_STRUCT_PBUF);
      LWIP_ASSERT("pbuf_alloc: pbuf q->payload properly aligned",
              ((mem_ptr_t)q->payload % MEM_ALIGNMENT) == 0);
      LWIP_ASSERT("check p->payload + p->len does not overflow pbuf",
                  ((u8_t*)p->payload + p->len <=
                   (u8_t*)p + SIZEOF_STRUCT_PBUF + PBUF_POOL_BUFSIZE_ALIGNED));
      q->ref = 1;
      /* calculate remaining length to be allocated */
      rem_len -= q->len;
      /* remember this pbuf for linkage in next iteration */
      r = q;
    }
    /* end of chain */
    /*r->next = NULL;*/

    break;

  /********************************************************/
  case PBUF_RAM: /*PBUF类型内存，一次性分配size大小的连续内存*/
    /* If pbuf is to be allocated in RAM, allocate memory for it. */
    p = (struct pbuf*)mem_malloc(LWIP_MEM_ALIGN_SIZE(SIZEOF_STRUCT_PBUF + offset) + LWIP_MEM_ALIGN_SIZE(length));
    /*PBUF_RAM由内存堆分配内存，此处调用了mem_malloc()，为内存堆分配函数，在mem.c文件中。*/
    /*分配的空间大小包括：pbuf结构头大小size_struct_pbuf，需要的数据存储空间大小length。*/
    /*还有一个offset，即有效数据起始偏移位置 */
    if (p == NULL) {
      return NULL;
    }
    /* Set up internal structure of the pbuf. */
    p->payload = LWIP_MEM_ALIGN((void *)((u8_t *)p + SIZEOF_STRUCT_PBUF + offset));
    p->len = p->tot_len = length;
    p->next = NULL;
    p->type = type;

    LWIP_ASSERT("pbuf_alloc: pbuf->payload properly aligned",
           ((mem_ptr_t)p->payload % MEM_ALIGNMENT) == 0);
    break;

  /********************************************************/
  /* pbuf references existing (non-volatile static constant) ROM payload? */
  case PBUF_ROM: /*ROM只需要分配小小的管理pbuf的控制管理内存*/
  /* pbuf references existing (externally allocated) RAM payload? */
  case PBUF_REF:
    /* only allocate memory for the pbuf structure */
    /*只需申请pbufs头控制结构体所需内存即可[luther.gliethttp]*/
    p = (struct pbuf *)memp_malloc(MEMP_PBUF);
    /*分配的内存池类型为MEMP_PBUF，大小恰为一个pbuf头的大小*/
    if (p == NULL) {
      LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_LEVEL_SERIOUS,
                  ("pbuf_alloc: Could not allocate MEMP_PBUF for PBUF_%s.\n",
                  (type == PBUF_ROM) ? "ROM" : "REF"));
      return NULL;
    }
    /* caller must set this field properly, afterwards */
    p->payload = NULL;
    p->len = p->tot_len = length;
    p->next = NULL;
    p->type = type;
    break;
   /********************************************************/
  default:
    LWIP_ASSERT("pbuf_alloc: erroneous type", 0);
    return NULL;
  }
  /* set reference count */
  p->ref = 1;
  /* set flags */
  p->flags = 0;
  LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_alloc(length=%"U16_F") == %p\n", length, (void *)p));
  return p;
}

三、连接pbufs

连接pbufs主要使用了三个函数：pbuf_cat()、pbuf_ref()、pbuf_chain()。

1、pbuf_ref()：将 pbuf中的 ref 加1

/**
 * Increment the reference count of the pbuf
 * 增量的pbuf的引用次数
 * @param p pbuf to increase reference counter of
 * 参数p的pbuf引用计数增加
 * /
void
pbuf_ref(struct pbuf *p)
{
    SYS_ARCH_DECL_PROTECT(old_level);
    /* pbuf given? */
    if(p != NULL) {
    SYS_ARCH_PROTECT(old_level);
    ++(p->ref); /*当pbuf不为空时将当前pbuf的ref加1*/
    SYS_ARCH_UNPROTECT(old_level);
    }
}

　　ref是pbuf结构中定义的一个16位无符号整数，包含一个引用计数，表示该pbuf被引用的次数，初始化一个pbuf的时候，ref字段值被设置为1，当有其他pbuf的next值针指向该pbuf时，该pbuf的字段值加1，所以要删除一个pbuf时，ref的值必须为1才能删除成功，否则删除失败。 统计有多少个指针指向这个pbuf。这些指针可能是应用程序的指针，协议栈自己的指针或者数据链中的pbuf->next指针，ref为0时，才可以释放pbuf

2、pbuf_cat()

/**
 * Concatenate two pbufs (each may be a pbuf chain) and take over
 * the caller's reference of the tail pbuf.
 * 连接两个pbufs(每个可能都是链表)并替代引用的呼叫方的尾部pbuf
 * @note The caller MAY NOT reference the tail pbuf afterwards.
 * 注意对方可能之后不提及尾部的pbuf
 * Use pbuf_chain() for that purpose.
 * 目的是使用pbuf_chain()，在pbuf_chain()之中使用这个函数来连接两个pbufs
 * @see pbuf_chain()
 */
void
pbuf_cat(struct pbuf *h, struct pbuf *t)
{
    struct pbuf *p; /* p最为一个中间变量 */
    LWIP_ERROR("(h != NULL) && (t != NULL)  (programmer violates API)",
    ((h != NULL) && (t != NULL)), return;);

    /* proceed to last pbuf of chain */
    for(p = h; p->next != NULL; p = p->next){
        /*使 p = h，当p的next指针(即h的next指针)不是空的时候，即后面还有pbuf*/
        /* add total length of second chain to all totals of first chain */
        /*第二个链的总长度加到第一个链上的总和*/
        p->tot_len += t->tot_len;
    }
    /* { p is last pbuf of first h chain, p->next == NULL } */
    /* p是第一个 h链的末尾pbuf，p->next为空，也就是p的后面再也没有pbuf*/
    LWIP_ASSERT("p->tot_len == p->len (of last pbuf in chain)", p->tot_len == p->len);
    /* 我还没找到LWIP_ASSERT()是什么，感觉是在声明注释说的意思。有知道的请告诉我 */
    LWIP_ASSERT("p->next == NULL", p->next == NULL);
    /* add total length of second chain to last pbuf total of first chain */
    p->tot_len += t->tot_len;
    /* chain last pbuf of head (p) with first of tail (t) */
    /* 连接后一个pbuf的头(即 p的头)和第一个的尾部(即 t的尾部)*/
    p->next = t;
    /* p->next now references t, but the caller will drop its reference to t,
    * so netto there is no change to the reference count of t.
    * p的next指针现在引用t，但是呼叫方将放弃引用t。所以netto没有改变t的引用计数
    */
}

3、pbuf_chain()

/**
 * Chain two pbufs (or pbuf chains) together.
 * 连接两个pbufs(或者pbufs链表)在一起
 * The caller MUST call pbuf_free(t) once it has stopped
 * using it. Use pbuf_cat() instead if you no longer use t.
 * 呼叫方必须调用pbuf_free（t）一旦终止使用它。使用pbuf_cat()而不是你不在使用t
 * @param h head pbuf (chain)
 * @param t tail pbuf (chain)
 * @note The pbufs MUST belong to the same packet.
 * @note MAY NOT be called on a packet queue.
 *
 * The ->tot_len fields of all pbufs of the head chain are adjusted.
 * The ->next field of the last pbuf of the head chain is adjusted.
 * The ->ref field of the first pbuf of the tail chain is adjusted.
 *
 */
void
pbuf_chain(struct pbuf *h, struct pbuf *t)
{
  pbuf_cat(h, t); //将 h和 t进行连接
  /* t is now referenced by h    t现在被h引用*/
  pbuf_ref(t); /* 将t的ref加1，也就是t被引用了一次*/
  LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_chain: %p references %p\n", (void *)h, (void *)t));
}

4、pbuf的释放函数pbuf_free()

　　pbuf的申请主要是通过两种方式实现的，即内存池分配与内存堆分配而得到的，所以pbuf的释放也要按照两种方式进行。
　　在删除一个pbuf结构之前，首先要检查这个pbuf是那种类型的，根据类型的不同，使用不同的内存释放函数进程删除，即内存堆释放函数或者内存池释放函数。PBUF_POOL、PBUF_ROM、PBUF_REF是由内存池分配的，需要调用memp_free()函数进行删除，而PBUF_RAM是由内存堆分配的，需要调用mem_free()函数进行删除。

/**
 * Dereference a pbuf chain or queue and deallocate any no-longer-used
 * pbufs at the head of this chain or queue.
 *
 * Decrements the pbuf reference count. If it reaches zero, the pbuf is
 * deallocated.
 *
 * For a pbuf chain, this is repeated for each pbuf in the chain,
 * up to the first pbuf which has a non-zero reference count after
 * decrementing. So, when all reference counts are one, the whole
 * chain is free'd.
 *
 * @param p The pbuf (chain) to be dereferenced.
 *
 * @return the number of pbufs that were de-allocated
 * from the head of the chain.
 *
 * @note MUST NOT be called on a packet queue (Not verified to work yet).
 * @note the reference counter of a pbuf equals the number of pointers
 * that refer to the pbuf (or into the pbuf).
 *
 * @internal examples:
 *
 * Assuming existing chains a->b->c with the following reference
 * counts, calling pbuf_free(a) results in:
 *
 * 1->2->3 becomes ...1->3
 * 3->3->3 becomes 2->3->3
 * 1->1->2 becomes ......1
 * 2->1->1 becomes 1->1->1
 * 1->1->1 becomes .......
 *
 */
u8_t
pbuf_free(struct pbuf *p)
{
  u16_t type;
  struct pbuf *q;
  u8_t count;

  if (p == NULL) {
    LWIP_ASSERT("p != NULL", p != NULL);
    /* if assertions are disabled, proceed with debug output */
    LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_LEVEL_SERIOUS,
      ("pbuf_free(p == NULL) was called.\n"));
    return 0;
  }
  LWIP_DEBUGF(PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_free(%p)\n", (void *)p));

  PERF_START;

  LWIP_ASSERT("pbuf_free: sane type",
    p->type == PBUF_RAM || p->type == PBUF_ROM ||
    p->type == PBUF_REF || p->type == PBUF_POOL);

  count = 0;
  /* de-allocate all consecutive pbufs from the head of the chain that
   * obtain a zero reference count after decrementing*/
  while (p != NULL) {
    u16_t ref;
    SYS_ARCH_DECL_PROTECT(old_level);
    /* Since decrementing ref cannot be guaranteed to be a single machine operation
     * we must protect it. We put the new ref into a local variable to prevent
     * further protection. */
    SYS_ARCH_PROTECT(old_level);
    /* all pbufs in a chain are referenced at least once */
    LWIP_ASSERT("pbuf_free: p->ref > 0", p->ref > 0);
    /* decrease reference count (number of pointers to pbuf) */
    ref = --(p->ref);
    SYS_ARCH_UNPROTECT(old_level);
    /* this pbuf is no longer referenced to? */
    if (ref == 0) {
      /* remember next pbuf in chain for next iteration */
      q = p->next;
      LWIP_DEBUGF( PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_free: deallocating %p\n", (void *)p));
      type = p->type;
#if LWIP_SUPPORT_CUSTOM_PBUF
      /* is this a custom pbuf? */
      if ((p->flags & PBUF_FLAG_IS_CUSTOM) != 0) {
        struct pbuf_custom *pc = (struct pbuf_custom*)p;
        LWIP_ASSERT("pc->custom_free_function != NULL", pc->custom_free_function != NULL);
        pc->custom_free_function(p);
      } else
#endif /* LWIP_SUPPORT_CUSTOM_PBUF */
      {
        /* is this a pbuf from the pool? */
        if (type == PBUF_POOL) {
          /*PBUF_POOL是MEMP_PBUF_POOL类型的，由内存池分配，调用memp_free()*/
          memp_free(MEMP_PBUF_POOL, p);
        /* is this a ROM or RAM referencing pbuf? */
        } else if (type == PBUF_ROM || type == PBUF_REF) {
          /*PBUF_ROM和PBUF_REF是MEMP_PBUF类型的，由内存池分配，调用memp_free()*/
          memp_free(MEMP_PBUF, p);
        /* type == PBUF_RAM */
        } else {
          /*PBUF_RAM是由内存堆分配的，调用mem_free进行释放*/
          mem_free(p);
        }
      }
      count++;
      /* proceed to next pbuf */
      p = q;
    /* p->ref > 0, this pbuf is still referenced to */
    /* (and so the remaining pbufs in chain as well) */
    } else {
      LWIP_DEBUGF( PBUF_DEBUG | LWIP_DBG_TRACE, ("pbuf_free: %p has ref %"U16_F", ending here.\n", (void *)p, ref));
      /* stop walking through the chain */
      p = NULL;
    }
  }
  PERF_STOP("pbuf_free");
  /* return number of de-allocated pbufs */
  return count;
}