Tensorflow的CNN教程解析

之前的博客我们已经对RNN模型有了个粗略的了解。作为一个时序性模型，RNN的强大不需要我在这里重复了。今天，让我们来看看除了RNN外另一个特殊的，同时也是广为人知的强大的神经网络模型，即CNN模型。今天的讨论主要是基于Tensorflow的CIFAR10教程，不过作为对比，我们也会对Tensorflow的MINST教程作解析以及对比。很快大家就会发现，逻辑上考虑，其实内容都是大同小异的。由于所对应的目标不一样，在数据处理方面可能存在着些许差异，这里我们以CIFAR10的为基准，有兴趣的朋友欢迎去阅读并学习MNIST的过程，地址点击这里。CIFAR10的英文教程在Tensorflow官网上可以获得，教程代码地址点击这里。

CNN简介

CNN是一个神奇的深度学习框架，也是深度学习学科里的一个异类。在被誉为AI寒冬的90年末到2000年初，在大部分学者都弃坑的情况下，CNN的效用却不减反增，感谢Yann LeCun！CNN的架构其实很符合其名，Convolutional Neural Network，CNN在运做的开始运用了卷积（convolution）的概念，外加pooling等方式在多次卷积了图像并形成多个特征图后，输入被平铺开进入一个完全连接的多层神经网络里（fully connected network）里，并由输出的softmax来判断图片的分类情况。该框架的发展史也很有趣，早在90年代末，以LeCun命名的Le-Net5就已经闻名。在深度学习火热后，更多的框架变种也接踵而至，较为闻名的包括多伦多大学的AlexNet，谷歌的GoogLeNet，牛津的OxfordNet外还有Network in Network（NIN），VGG16等多个network。最近，对物体识别的研究开发了RCNN框架，可见在深度学习发展迅猛的今天，CNN框架依然是很多著名研究小组的课题，特别是在了解了Alpha-Go的运作里也可以看到CNN的身影，可见其能力！至于CNN模型的基础构架，这方面的资源甚多，就不一一列举了。

CIFAR10代码分析

在运行CIFAR10代码时，你只需要下载该代码，然后cd到代码目录后直接输入python cifar10_train.py就可以了。默认的迭代步骤为100万步，每一步骤需要约3~4秒，运行5小时可以完成近10万步。由于根据cifar10_train.py的描述10万步的准确率为86%左右，我们运行近5个小时左右就可以了，没必要运行全部的100万步。查看结果时，运行python cifar_10_eval.py就可以了。由于模型被存储在了tmp目录里，eval文件可以找寻到最近保存的模型并运行该模型，所以还是很方便的。这个系统在运行后可以从照片里识别10种不同的物体，包括飞机等。这么好玩的系统，快让我们来看一看是怎么实现的吧！

首先，让我们来看下cifar1_train.py文件。文件里的核心为train函数，它的表现如下：

def train():
  """Train CIFAR-10 for a number of steps."""
  with tf.Graph().as_default():
    global_step = tf.Variable(0, trainable=False)

    # Get images and labels for CIFAR-10.
    # 输入选用的是distored_inputs函数
    images, labels = cifar10.distorted_inputs()

    # Build a Graph that computes the logits predictions from the
    # inference model.
    logits = cifar10.inference(images)

    # Calculate loss.
    loss = cifar10.loss(logits, labels)

    # Build a Graph that trains the model with one batch of examples and
    # updates the model parameters.
    train_op = cifar10.train(loss, global_step)

    # Create a saver.
    saver = tf.train.Saver(tf.all_variables())

    # Build the summary operation based on the TF collection of Summaries.
    summary_op = tf.merge_all_summaries()

    # Build an initialization operation to run below.
    init = tf.initialize_all_variables()

    # Start running operations on the Graph.
    sess = tf.Session(config=tf.ConfigProto(
        log_device_placement=FLAGS.log_device_placement))
    sess.run(init)

    # Start the queue runners.
    tf.train.start_queue_runners(sess=sess)

    summary_writer = tf.train.SummaryWriter(FLAGS.train_dir, sess.graph)

    # 在最高的迭代步骤数里进行循环迭代
    for step in xrange(FLAGS.max_steps):
      start_time = time.time()
      _, loss_value = sess.run([train_op, loss])
      duration = time.time() - start_time

      assert not np.isnan(loss_value), 'Model diverged with loss = NaN'
      # 每10个输入数据显示次step，loss，时间等运行数据
      if step % 10 == 0:
        num_examples_per_step = FLAGS.batch_size
        examples_per_sec = num_examples_per_step / duration
        sec_per_batch = float(duration)

        format_str = ('%s: step %d, loss = %.2f (%.1f examples/sec; %.3f '
                      'sec/batch)')
        print (format_str % (datetime.now(), step, loss_value,
                             examples_per_sec, sec_per_batch))
      # 每100个输入数据将网络的状况体现在summary里
      if step % 100 == 0:
        summary_str = sess.run(summary_op)
        summary_writer.add_summary(summary_str, step)

      # Save the model checkpoint periodically.
      # 每1000个输入数据保存次模型
      if step % 1000 == 0 or (step + 1) == FLAGS.max_steps:
        checkpoint_path = os.path.join(FLAGS.train_dir, 'model.ckpt')
        saver.save(sess, checkpoint_path, global_step=step)

这个训练函数本身逻辑很清晰，除了它运用了大量的cifar10.py文件里的函数外，一个值得注意的地方是输入里应用的是distorded_inputs函数。这个很有意思，因为据论文表达，对输入数据进行一定的处理后可以得到新的数据，这是增加数据存储量的一个简便的方法，那么具体它是如何做到的呢?让我们来看看这个distorded_inputs函数。在cifar10.py文件里，distorded_inputs函数实质上是一个wrapper，包装了来自cifar10_input.py函数里的distorted_inputs()函数。这个函数的逻辑如下：

def distorted_inputs(data_dir, batch_size):
  """Construct distorted input for CIFAR training using the Reader ops.
  Args:
    data_dir: Path to the CIFAR-10 data directory.
    batch_size: Number of images per batch.
  Returns:
    images: Images. 4D tensor of [batch_size, IMAGE_SIZE, IMAGE_SIZE, 3] size.
    labels: Labels. 1D tensor of [batch_size] size.
  """
  filenames = [os.path.join(data_dir, 'data_batch_%d.bin' % i)
               for i in xrange(1, 6)]
  for f in filenames:
    if not tf.gfile.Exists(f):
      raise ValueError('Failed to find file: ' + f)

  # Create a queue that produces the filenames to read.
  filename_queue = tf.train.string_input_producer(filenames)

  # Read examples from files in the filename queue.
  read_input = read_cifar10(filename_queue)
  reshaped_image = tf.cast(read_input.uint8image, tf.float32)

  height = IMAGE_SIZE
  width = IMAGE_SIZE

  # Image processing for training the network. Note the many random
  # distortions applied to the image.

  # Randomly crop a [height, width] section of the image.
  # 步骤1：随机截取一个以[高，宽]为大小的图矩阵。
  distorted_image = tf.random_crop(reshaped_image, [height, width, 3])

  # Randomly flip the image horizontally.
  # 步骤2：随机颠倒图片的左右。概率为50%
  distorted_image = tf.image.random_flip_left_right(distorted_image)

  # Because these operations are not commutative, consider randomizing
  # the order their operation.
  #  步骤3：随机改变图片的亮度以及色彩对比。
  distorted_image = tf.image.random_brightness(distorted_image,
                                               max_delta=63)
  distorted_image = tf.image.random_contrast(distorted_image,
                                             lower=0.2, upper=1.8)

  # Subtract off the mean and divide by the variance of the pixels.
  float_image = tf.image.per_image_whitening(distorted_image)

  # Ensure that the random shuffling has good mixing properties.
  min_fraction_of_examples_in_queue = 0.4
  min_queue_examples = int(NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN *
                           min_fraction_of_examples_in_queue)
  print ('Filling queue with %d CIFAR images before starting to train. '
         'This will take a few minutes.' % min_queue_examples)

  # Generate a batch of images and labels by building up a queue of examples.
  return _generate_image_and_label_batch(float_image, read_input.label,
                                         min_queue_examples, batch_size,
                                         shuffle=True)

这里每一张图片被随机的截取一片图后有一定的概率被翻转，改变亮度对比等步骤。另外，最后一段的意思为在queue里有了不少于40%的数据的时候训练才能开始。那么在测试的时候，我们需要经过这个步骤么？答案是非也。在cifar10_input.py文件里，distorded_inputs函数的下方，一个名为inputs的函数代表了输入被运用在eval时的逻辑。在输入参数方面，这个inputs函数在保留了distorded_inputs的同时增加了一个名为eval_data的参数，一个bool参数代表了是运用训练的数据还是测试的数据。下面，让我们来大概看下这个函数的逻辑。

def inputs(eval_data, data_dir, batch_size):
  """Construct input for CIFAR evaluation using the Reader ops.
  Args:
    eval_data: bool, indicating if one should use the train or eval data set.
    data_dir: Path to the CIFAR-10 data directory.
    batch_size: Number of images per batch.
  Returns:
    images: Images. 4D tensor of [batch_size, IMAGE_SIZE, IMAGE_SIZE, 3] size.
    labels: Labels. 1D tensor of [batch_size] size.
  """
  if not eval_data:
    filenames = [os.path.join(data_dir, 'data_batch_%d.bin' % i)
                 for i in xrange(1, 6)]
    num_examples_per_epoch = NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN
  else:
    filenames = [os.path.join(data_dir, 'test_batch.bin')]
    num_examples_per_epoch = NUM_EXAMPLES_PER_EPOCH_FOR_EVAL

  for f in filenames:
    if not tf.gfile.Exists(f):
      raise ValueError('Failed to find file: ' + f)

  # Create a queue that produces the filenames to read.
  filename_queue = tf.train.string_input_producer(filenames)

  # Read examples from files in the filename queue.
  read_input = read_cifar10(filename_queue)
  reshaped_image = tf.cast(read_input.uint8image, tf.float32)

  height = IMAGE_SIZE
  width = IMAGE_SIZE

  # Image processing for evaluation.
  # Crop the central [height, width] of the image.
  # 截取图片中心区域
  resized_image = tf.image.resize_image_with_crop_or_pad(reshaped_image,
                                                         width, height)

  # Subtract off the mean and divide by the variance of the pixels.
  # 平衡图片的色差
  float_image = tf.image.per_image_whitening(resized_image)

  # Ensure that the random shuffling has good mixing properties.
  min_fraction_of_examples_in_queue = 0.4
  min_queue_examples = int(num_examples_per_epoch *
                           min_fraction_of_examples_in_queue)

  # Generate a batch of images and labels by building up a queue of examples.
  return _generate_image_and_label_batch(float_image, read_input.label,
                                         min_queue_examples, batch_size,
                                         shuffle=False)

这里，我们看到截取只有图片的中心，另外处理也只有平衡色差。但是，聪明的读者朋友一定能想到，如果一张关于飞机的图片是以飞机头为图片中心的，而训练集合里所有的飞机图片都是以机翼为图片中心的话，我们之前的distorded_inputs函数将有机会截取飞机头的区域，从而给我们的测试图片提供相似信息。另外，随机调整色差也包含了平均色差，所以我们的训练集实质上包含了更广，更多种的可能性，故可想而之会有机会得到更好的效果。

那么，讲了关于输入的小窍门，我们应该来看看具体的CNN模型了。如何制造一个CNN模型呢？让我们先来看一个简单的版本，即MNIST教程里的模型：

  # The variables below hold all the trainable weights. They are passed an
  # initial value which will be assigned when we call:
  # {tf.initialize_all_variables().run()}
  conv1_weights = tf.Variable(
      tf.truncated_normal([5, 5, NUM_CHANNELS, 32],  # 5x5 filter, depth 32.
                          stddev=0.1,
                          seed=SEED, dtype=data_type()))
  conv1_biases = tf.Variable(tf.zeros([32], dtype=data_type()))
  conv2_weights = tf.Variable(tf.truncated_normal(
      [5, 5, 32, 64], stddev=0.1,
      seed=SEED, dtype=data_type()))
  conv2_biases = tf.Variable(tf.constant(0.1, shape=[64], dtype=data_type()))
  fc1_weights = tf.Variable(  # fully connected, depth 512.
      tf.truncated_normal([IMAGE_SIZE // 4 * IMAGE_SIZE // 4 * 64, 512],
                          stddev=0.1,
                          seed=SEED,
                          dtype=data_type()))
  fc1_biases = tf.Variable(tf.constant(0.1, shape=[512], dtype=data_type()))
  fc2_weights = tf.Variable(tf.truncated_normal([512, NUM_LABELS],
                                                stddev=0.1,
                                                seed=SEED,
                                                dtype=data_type()))
  fc2_biases = tf.Variable(tf.constant(
      0.1, shape=[NUM_LABELS], dtype=data_type()))

  # We will replicate the model structure for the training subgraph, as well
  # as the evaluation subgraphs, while sharing the trainable parameters.
  def model(data, train=False):
    """The Model definition."""
    # 2D convolution, with 'SAME' padding (i.e. the output feature map has
    # the same size as the input). Note that {strides} is a 4D array whose
    # shape matches the data layout: [image index, y, x, depth].
    conv = tf.nn.conv2d(data,
                        conv1_weights,
                        strides=[1, 1, 1, 1],
                        padding='SAME')
    # Bias and rectified linear non-linearity.
    relu = tf.nn.relu(tf.nn.bias_add(conv, conv1_biases))
    # Max pooling. The kernel size spec {ksize} also follows the layout of
    # the data. Here we have a pooling window of 2, and a stride of 2.
    pool = tf.nn.max_pool(relu,
                          ksize=[1, 2, 2, 1],
                          strides=[1, 2, 2, 1],
                          padding='SAME')
    conv = tf.nn.conv2d(pool,
                        conv2_weights,
                        strides=[1, 1, 1, 1],
                        padding='SAME')
    relu = tf.nn.relu(tf.nn.bias_add(conv, conv2_biases))
    pool = tf.nn.max_pool(relu,
                          ksize=[1, 2, 2, 1],
                          strides=[1, 2, 2, 1],
                          padding='SAME')
    # Reshape the feature map cuboid into a 2D matrix to feed it to the
    # fully connected layers.
    pool_shape = pool.get_shape().as_list()
    reshape = tf.reshape(
        pool,
        [pool_shape[0], pool_shape[1] * pool_shape[2] * pool_shape[3]])
    # Fully connected layer. Note that the '+' operation automatically
    # broadcasts the biases.
    hidden = tf.nn.relu(tf.matmul(reshape, fc1_weights) + fc1_biases)
    # Add a 50% dropout during training only. Dropout also scales
    # activations such that no rescaling is needed at evaluation time.
    if train:
      hidden = tf.nn.dropout(hidden, 0.5, seed=SEED)
    return tf.matmul(hidden, fc2_weights) + fc2_biases

  # Training computation: logits + cross-entropy loss.
  logits = model(train_data_node, True)
  loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(
      logits, train_labels_node))

  # L2 regularization for the fully connected parameters.
  regularizers = (tf.nn.l2_loss(fc1_weights) + tf.nn.l2_loss(fc1_biases) +
                  tf.nn.l2_loss(fc2_weights) + tf.nn.l2_loss(fc2_biases))
  # Add the regularization term to the loss.
  loss += 5e-4 * regularizers

  # Optimizer: set up a variable that's incremented once per batch and
  # controls the learning rate decay.
  batch = tf.Variable(0, dtype=data_type())
  # Decay once per epoch, using an exponential schedule starting at 0.01.
  learning_rate = tf.train.exponential_decay(
      0.01,                # Base learning rate.
      batch * BATCH_SIZE,  # Current index into the dataset.
      train_size,          # Decay step.
      0.95,                # Decay rate.
      staircase=True)
  # Use simple momentum for the optimization.
  optimizer = tf.train.MomentumOptimizer(learning_rate,
                                         0.9).minimize(loss,
                                                       global_step=batch)

  # Predictions for the current training minibatch.
  train_prediction = tf.nn.softmax(logits)

  # Predictions for the test and validation, which we'll compute less often.
  eval_prediction = tf.nn.softmax(model(eval_data))

这段代码很直白，在定义了convolution1,convolution2,fully_connected1和fully_connected2层神经网络的weight和biases参数后，在模型函数里，我们通过conv2d, relu, max_pool等方式在两次重复后将得到的结果重新整理后输入那个fully connected的神经网络中，即matmul(reshape,fc1_weights) + fc1_biases。之后再经历了第二层的fully connected net后得到logits。定义loss以及optimizer等常见的过程后结果是由softmax来取得。这个逻辑我们在CIFAR10里也会见到，它的表达如下：

def inference(images):
  """Build the CIFAR-10 model.
  Args:
    images: Images returned from distorted_inputs() or inputs().
  Returns:
    Logits.
  """
  # We instantiate all variables using tf.get_variable() instead of
  # tf.Variable() in order to share variables across multiple GPU training runs.
  # If we only ran this model on a single GPU, we could simplify this function
  # by replacing all instances of tf.get_variable() with tf.Variable().
  #
  # conv1
  with tf.variable_scope('conv1') as scope:
    # 输入的图片由于是彩图，有三个channel，所以在conv2d中，我们规定
    # 输出为64个channel的feature map。
    kernel = _variable_with_weight_decay('weights', shape=[5, 5, 3, 64],
                                         stddev=1e-4, wd=0.0)
    conv = tf.nn.conv2d(images, kernel, [1, 1, 1, 1], padding='SAME')
    biases = _variable_on_cpu('biases', [64], tf.constant_initializer(0.0))
    bias = tf.nn.bias_add(conv, biases)
    conv1 = tf.nn.relu(bias, name=scope.name)
    _activation_summary(conv1)

  # pool1
  pool1 = tf.nn.max_pool(conv1, ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1],
                         padding='SAME', name='pool1')
  # norm1
  norm1 = tf.nn.lrn(pool1, 4, bias=1.0, alpha=0.001 / 9.0, beta=0.75,
                    name='norm1')

  # conv2
  with tf.variable_scope('conv2') as scope:
    # 由于之前的输出是64个channel，即我们这里的输入，我们的shape就会
    # 是输入channel数为64，输出，我们也规定为64
    kernel = _variable_with_weight_decay('weights', shape=[5, 5, 64, 64],
                                         stddev=1e-4, wd=0.0)
    conv = tf.nn.conv2d(norm1, kernel, [1, 1, 1, 1], padding='SAME')
    biases = _variable_on_cpu('biases', [64], tf.constant_initializer(0.1))
    bias = tf.nn.bias_add(conv, biases)
    conv2 = tf.nn.relu(bias, name=scope.name)
    _activation_summary(conv2)

  # norm2
  norm2 = tf.nn.lrn(conv2, 4, bias=1.0, alpha=0.001 / 9.0, beta=0.75,
                    name='norm2')
  # pool2
  pool2 = tf.nn.max_pool(norm2, ksize=[1, 3, 3, 1],
                         strides=[1, 2, 2, 1], padding='SAME', name='pool2')

  # local3
  with tf.variable_scope('local3') as scope:
    # Move everything into depth so we can perform a single matrix multiply.
    reshape = tf.reshape(pool2, [FLAGS.batch_size, -1])
    dim = reshape.get_shape()[1].value
    # 这里之前在reshape时的那个-1是根据tensor的大小自动定义为batch_size和
    # 剩下的，所以我们剩下的就是一张图的所有内容，我们将它训练并map到384
    # 个神经元节点上
    weights = _variable_with_weight_decay('weights', shape=[dim, 384],
                                          stddev=0.04, wd=0.004)
    biases = _variable_on_cpu('biases', [384], tf.constant_initializer(0.1))
    local3 = tf.nn.relu(tf.matmul(reshape, weights) + biases, name=scope.name)
    _activation_summary(local3)

  # local4
  with tf.variable_scope('local4') as scope:
    #由于我们之前的节点有384个，这里我们进一步缩减为192个。
    weights = _variable_with_weight_decay('weights', shape=[384, 192],
                                          stddev=0.04, wd=0.004)
    biases = _variable_on_cpu('biases', [192], tf.constant_initializer(0.1))
    local4 = tf.nn.relu(tf.matmul(local3, weights) + biases, name=scope.name)
    _activation_summary(local4)

  # softmax, i.e. softmax(WX + b)
  with tf.variable_scope('softmax_linear') as scope:
    # 这是softmax输出时的网络，我们由192个节点map到输出的不同数量上，这里假设
    # 有10类，我们就输出10个num_classes。
    weights = _variable_with_weight_decay('weights', [192, NUM_CLASSES],
                                          stddev=1/192.0, wd=0.0)
    biases = _variable_on_cpu('biases', [NUM_CLASSES],
                              tf.constant_initializer(0.0))
    softmax_linear = tf.add(tf.matmul(local4, weights), biases, name=scope.name)
    _activation_summary(softmax_linear)

  return softmax_linear

这里的逻辑跟之前的在框架上基本一样，不同在哪里呢？首先，这次我们的输入是彩图。学过图片处理的朋友肯定知道彩图有3个channel，而之前MNIST只是单个channel的灰白图。所以，在我们制作feature map的时候，由1个channel map到了32个（注，那个NUM_CHANNELS是1）。这里我们不过把NUM_CHANNELS给直接写为了3而已。另外，我们还运用了variable scope，这是一种很好的方式来界定何时对那些变量进行分享，同时，我们也不需要反复定义weight和biases的名字了。

对Loss的定义由loss函数写明，其内容无非是运用了sparse_softmax_corss_entropy_with_logits，基本流程同于MNIST，这里将不详细描述。最后，cifar10.py里的train函数虽然逻辑很简单，但是也有值得注意的地方。代码如下：

def train(total_loss, global_step):
  """Train CIFAR-10 model.
  Create an optimizer and apply to all trainable variables. Add moving
  average for all trainable variables.
  Args:
    total_loss: Total loss from loss().
    global_step: Integer Variable counting the number of training steps
      processed.
  Returns:
    train_op: op for training.
  """
  # Variables that affect learning rate.
  num_batches_per_epoch = NUM_EXAMPLES_PER_EPOCH_FOR_TRAIN / FLAGS.batch_size
  decay_steps = int(num_batches_per_epoch * NUM_EPOCHS_PER_DECAY)

  # Decay the learning rate exponentially based on the number of steps.
  lr = tf.train.exponential_decay(INITIAL_LEARNING_RATE,
                                  global_step,
                                  decay_steps,
                                  LEARNING_RATE_DECAY_FACTOR,
                                  staircase=True)
  tf.scalar_summary('learning_rate', lr)

  # Generate moving averages of all losses and associated summaries.
  loss_averages_op = _add_loss_summaries(total_loss)

  # Compute gradients.
  # control dependencies的运用。这里只有loss_averages_op完成了
  # 我们才会进行gradient descent的优化。
  with tf.control_dependencies([loss_averages_op]):
    opt = tf.train.GradientDescentOptimizer(lr)
    grads = opt.compute_gradients(total_loss)

  # Apply gradients.
  apply_gradient_op = opt.apply_gradients(grads, global_step=global_step)

  # Add histograms for trainable variables.
  for var in tf.trainable_variables():
    tf.histogram_summary(var.op.name, var)

  # Add histograms for gradients.
  for grad, var in grads:
    if grad is not None:
      tf.histogram_summary(var.op.name + '/gradients', grad)

  # Track the moving averages of all trainable variables.
  variable_averages = tf.train.ExponentialMovingAverage(
      MOVING_AVERAGE_DECAY, global_step)
  variables_averages_op = variable_averages.apply(tf.trainable_variables())

  with tf.control_dependencies([apply_gradient_op, variables_averages_op]):
    train_op = tf.no_op(name='train')

  return train_op

这里多出的一些内容为收集网络运算时的一些临时结果，如记录所有的loss的loss_averages_op = _add_loss_summaries(total_loss)以及对参数的histogram：tf.histogram_summary(var.op.name, var)。值得注意的地方是这里多次地使用了control_dependency概念，即dependency条件没有达成前，dependency内的代码是不会运行的。这个概念在Tensorflow中有着重要的意义，这里是一个实例，给大家很好的阐述了这个概念，建议有兴趣的朋友可以多加研究。至此，图片的训练便到此为止。

那么eval文件是如何评价模型的好坏的呢？让我们来简单的看下eval文件的内容。我们首先通过evaluate函数中的cifar10.inputs函数得到输入图片以及其对应的label，之后，通过之前介绍的inference函数，即CNN框架得到logits，之后我们通过tensorflow的in_top_k函数来判断我们得到的那个logit是否在我们label里。这里的k被设置为1并对结果做展示以及记录等工作。有兴趣的朋友可以仔细阅读这段代码，这里将不详细说明。

至此，系统完成，我们对于如何建立一个CNN系统有了初步了解。