Convolutional Neural Networks (CNNs) are responsible for the major breakthroughs in image recognition made in the past few years. In this chapter we will cover:

  • Implementing a Simpler CNN
  • Implementing an Advanced CNN
  • Retraining Existing CNN models
  • Applying Stylenet/Neural-Style
  • Implementing DeepDream

The CIFAR-10 and CIFAR-100 are labeled subsets of the 80 million (八千万) tiny images dataset. They were collected by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.

Click here: https://www.cs.toronto.edu/~kriz/cifar.html

Visual dictionary

Click on top of the map to visualize the images in that region of the visual dictionary.

We present a visualization of all the nouns in the English language arranged by semantic meaning. 根据语义可视化名词

Each of the tiles in the mosaic is an arithmetic average of images relating to one of 53,464 nouns. 与五万多个名词之一相关的一个由一大堆图像算数平均后的像素点

The images for each word were obtained using Google's Image Search and other engines. A total of 7,527,697 images were used, each tile being the average of 140 images.

大概就是如此，如下：

The CIFAR-10 dataset

The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class.

There are training images and test images.

Here are the classes in the dataset, as well as 10 random images from each:

airplane
automobile
bird
cat
deer
dog
frog
horse
ship
truck

The classes are completely mutually exclusive. There is no overlap between automobiles and trucks. "Automobile" includes sedans, SUVs, things of that sort. "Truck" includes only big trucks. Neither includes pickup trucks.

This recipe is an adapted version of the official TensorFlow CIFAR-10 tutorial, which is
available under the See also section at the end of this chapter. We have condensed the
tutorial into one script and will go through it line-by-line and explain all the code that is
necessary. We also revert some constants and parameters to the original cited paper values,
which we will point out in the following appropriated steps.

前言：使用我的四核i7小笔记本训练了一次数据，CPU满负荷，生怕机器就此挂了。

Starting Training

Generation 50: Loss = 1.99133

Generation 100: Loss = 1.82077

Generation 150: Loss = 2.08245

Generation 200: Loss = 1.60232

Generation 250: Loss = 1.73661

Generation 300: Loss = 1.71986

Generation 350: Loss = 1.45252

Generation 400: Loss = 1.51505

Generation 450: Loss = 1.50190

Generation 500: Loss = 1.43989

 --- Test Accuracy = 49.22%.

Generation 550: Loss = 1.34899

Generation 600: Loss = 1.28325

Generation 650: Loss = 1.45376

Generation 700: Loss = 1.22179

Generation 750: Loss = 1.40790

Generation 800: Loss = 1.23635

Generation 850: Loss = 1.36577

Generation 900: Loss = 1.29193

Generation 950: Loss = 1.16195

Generation 1000: Loss = 1.30807

 --- Test Accuracy = 53.91%.

Generation 1050: Loss = 1.53120

Generation 1100: Loss = 1.19605

Generation 1150: Loss = 1.07220

Generation 1200: Loss = 1.03782

Generation 1250: Loss = 1.22976

Generation 1300: Loss = 0.96371

Generation 1350: Loss = 1.06199

Generation 1400: Loss = 1.04158

Generation 1450: Loss = 1.09863

Generation 1500: Loss = 1.00462

 --- Test Accuracy = 61.72%.

Generation 1550: Loss = 0.93589

Generation 1600: Loss = 0.94716

Generation 1650: Loss = 0.97767

Generation 1700: Loss = 0.89214

Generation 1750: Loss = 0.93194

Generation 1800: Loss = 0.78864

Generation 1850: Loss = 0.79083

Generation 1900: Loss = 1.16496

Generation 1950: Loss = 0.95690

Generation 2000: Loss = 0.71276

 --- Test Accuracy = 64.84%.

Generation 2050: Loss = 0.90579

Generation 2100: Loss = 0.82735

Generation 2150: Loss = 0.89798

Generation 2200: Loss = 0.90343

Generation 2250: Loss = 0.83713

Generation 2300: Loss = 0.85635

Generation 2350: Loss = 0.83437

Generation 2400: Loss = 0.83430

Generation 2450: Loss = 0.87104

Generation 2500: Loss = 0.84299

 --- Test Accuracy = 74.22%.

Generation 2550: Loss = 0.79991

Generation 2600: Loss = 0.80746

Generation 2650: Loss = 0.82027

Generation 2700: Loss = 0.84372

Generation 2750: Loss = 0.81977

Generation 2800: Loss = 0.77057

Generation 2850: Loss = 0.75629

Generation 2900: Loss = 0.82681

Generation 2950: Loss = 0.88289

Generation 3000: Loss = 0.94536

 --- Test Accuracy = 71.09%.

Generation 3050: Loss = 0.76870

Generation 3100: Loss = 0.80715

Generation 3150: Loss = 0.80056

Generation 3200: Loss = 0.78387

Generation 3250: Loss = 0.59328

Generation 3300: Loss = 0.84897

Generation 3350: Loss = 0.67461

Generation 3400: Loss = 0.64628

Generation 3450: Loss = 0.64160

Generation 3500: Loss = 0.63691

 --- Test Accuracy = 70.31%.

Generation 3550: Loss = 0.63177

Generation 3600: Loss = 0.74349

Generation 3650: Loss = 0.64307

Generation 3700: Loss = 0.61021

Generation 3750: Loss = 0.64688

Generation 3800: Loss = 0.63159

Generation 3850: Loss = 0.78472

Generation 3900: Loss = 0.75076

Generation 3950: Loss = 0.53717

Generation 4000: Loss = 0.46514

 --- Test Accuracy = 65.62%.

Generation 4050: Loss = 0.68460

Generation 4100: Loss = 0.58425

Generation 4150: Loss = 0.47215

Generation 4200: Loss = 0.58976

Generation 4250: Loss = 0.64681

Generation 4300: Loss = 0.77239

Generation 4350: Loss = 0.58956

Generation 4400: Loss = 0.70569

Generation 4450: Loss = 0.66185

Generation 4500: Loss = 0.46662

 --- Test Accuracy = 76.56%.

Generation 4550: Loss = 0.49475

Generation 4600: Loss = 0.54739

Generation 4650: Loss = 0.52838

Generation 4700: Loss = 0.81228

Generation 4750: Loss = 0.49100

Generation 4800: Loss = 0.51341

Generation 4850: Loss = 0.47875

Generation 4900: Loss = 0.37848

Generation 4950: Loss = 0.52750

Generation 5000: Loss = 0.53570

 --- Test Accuracy = 63.28%.

Generation 5050: Loss = 0.63138

Generation 5100: Loss = 0.49153

Generation 5150: Loss = 0.54037

Generation 5200: Loss = 0.72630

Generation 5250: Loss = 0.44166

Generation 5300: Loss = 0.51812

Generation 5350: Loss = 0.51912

Generation 5400: Loss = 0.54622

Generation 5450: Loss = 0.41648

Generation 5500: Loss = 0.57976

 --- Test Accuracy = 71.88%.

Generation 5550: Loss = 0.55666

Generation 5600: Loss = 0.44564

Generation 5650: Loss = 0.46812

... 未完，但担心机器挂了

部分日志

1. 初始化.

# More Advanced CNN Model: CIFAR-10

#---------------------------------------

#

# In this example, we will download the CIFAR-10 images

# and build a CNN model with dropout and regularization

#

# CIFAR is composed ot 50k train and 10k test

# images that are 32x32.

import os

import tarfile

import matplotlib.pyplot as plt

import tensorflow as tf

from six.moves import urllib

from tensorflow.python.framework import ops

ops.reset_default_graph()

# Change Directory

abspath = os.path.abspath(__file__)

dname = os.path.dirname(abspath)

os.chdir(dname)

# Start a graph session

sess = tf.Session()

2. 参数设置.

# Set model parameters

batch_size   = 128

data_dir     = 'temp'

output_every = 50

generations  = 20000

eval_every   = 500

image_height = 32

image_width  = 32

crop_height  = 24

crop_width   = 24

num_channels = 3

num_targets  = 10

extract_folder = 'cifar-10-batches-bin'

# Exponential Learning Rate Decay Params

learning_rate    = 0.1

lr_decay         = 0.1

num_gens_to_wait = 250.

# Extract model parameters

image_vec_length = image_height * image_width * num_channels

record_length    = 1 + image_vec_length # ( + 1 for the 0-9 label)

3. Create folder then load data. 

# Create folder then load data

data_dir = 'temp'

if not os.path.exists(data_dir):

    os.makedirs(data_dir)

cifar10_url = 'http://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz'

# Check if file exists, otherwise download it

data_file = os.path.join(data_dir, 'cifar-10-binary.tar.gz')

if os.path.isfile(data_file):

    pass

else:

    # Download file

    def progress(block_num, block_size, total_size):

        progress_info = [cifar10_url, float(block_num * block_size) / float(total_size) * 100.0]

        print('\r Downloading {} - {:.2f}%'.format(*progress_info), end="")

    filepath, _ = urllib.request.urlretrieve(cifar10_url, data_file, progress)

    # Extract file

    tarfile.open(filepath, 'r:gz').extractall(data_dir)

4. Get data then declare model.

# Get data

print('Getting/Transforming Data.')

# Initialize the data pipeline

images, targets = input_pipeline(batch_size, train_logical=True)　　// --> 4.1

# Get batch test images and targets from pipline

test_images, test_targets = input_pipeline(batch_size, train_logical=False)

# Declare Model

print('Creating the CIFAR10 Model.')

with tf.variable_scope('model_definition') as scope:

    # Declare the training network model

    model_output = cifar_cnn_model(images, batch_size)

    # This is very important!!!  We must set the scope to REUSE the variables,

    # Otherwise, when we set the test network model, it will create new random variables.

    # Otherwise we get random evaluations on the test batches.

    scope.reuse_variables()

    test_output = cifar_cnn_model(test_images, batch_size)

4.1 序列加载 - input_pipeline

# Create a CIFAR image pipeline from reader

def input_pipeline(batch_size, train_logical=True):

    if train_logical:

        files = [os.path.join(data_dir, extract_folder, 'data_batch_{}.bin'.format(i)) for i in range(1,6)]

    else:

        files = [os.path.join(data_dir, extract_folder, 'test_batch.bin')]

    filename_queue = tf.train.string_input_producer(files)

    image, label   = read_cifar_files(filename_queue)　　// --> 4.2

    # min_after_dequeue defines how big a buffer we will randomly sample

    #   from -- bigger means better shuffling but slower start up and more

    #   memory used.

    # capacity must be larger than min_after_dequeue and the amount larger

    #   determines the maximum we will prefetch.  Recommendation:

    #   min_after_dequeue + (num_threads + a small safety margin) * batch_size

    min_after_dequeue = 5000

    capacity = min_after_dequeue + 3 * batch_size

    example_batch, label_batch = tf.train.shuffle_batch([image, label],

                                                        batch_size=batch_size,

                                                        capacity=capacity,

                                                        min_after_dequeue=min_after_dequeue)

    return(example_batch, label_batch)

4.2 图片预处理 - read_cifar_files

对图片样本数据做了一些预处理工作。

# Define CIFAR reader

def read_cifar_files(filename_queue, distort_images = True):

    reader = tf.FixedLengthRecordReader(record_bytes=record_length)

    key, record_string = reader.read(filename_queue)

    record_bytes = tf.decode_raw(record_string, tf.uint8)

    image_label = tf.cast(tf.slice(record_bytes, [0], [1]), tf.int32)

    # Extract image

    image_extracted = tf.reshape(tf.slice(record_bytes, [1], [image_vec_length]),

                                 [num_channels, image_height, image_width])

    # Reshape image

    image_uint8image = tf.transpose(image_extracted, [1, 2, 0])

    reshaped_image = tf.cast(image_uint8image, tf.float32)

    # Randomly Crop image

    final_image = tf.image.resize_image_with_crop_or_pad(reshaped_image, crop_width, crop_height)

    # 一定的预处理步骤

    if distort_images:

        # Randomly flip the image horizontally, change the brightness and contrast

        final_image = tf.image.random_flip_left_right(final_image)

        final_image = tf.image.random_brightness(final_image,max_delta=63)

        final_image = tf.image.random_contrast(final_image,lower=0.2, upper=1.8)

    # Normalize whitening

    final_image = tf.image.per_image_standardization(final_image)

    return(final_image, image_label)

4.3 定义模型 

Next, we can declare our model function.

The model we will use has 2 convolutional layers, followed by 3 fully connected layers.

To make variable declaration easier, we'll start by declaring two variable functions. (ori: 32x32 --> 8x8)

  • The two convolutional layers will create 64 features each.
  • The 1st fully connected layer will connect the 2nd convolutional layer with 384 hidden nodes.
  • The 2nd fully connected operation will connect those 384 hidden nodes to 192 hidden nodes.　　
  • The final hidden layer operation will then connect the 192 nodes to the 10 output classes we are trying to predict.

【input --> 1st conv + pooling --> 2nd conv + pooling --> fully connected --> 384 -- fully connected --> 192 -- fully connected --> 10】

# Define the model architecture, this will return logits from images

def cifar_cnn_model(input_images, batch_size, train_logical=True):

    def truncated_normal_var(name, shape, dtype):

        return(tf.get_variable(name=name, shape=shape, dtype=dtype, initializer=tf.truncated_normal_initializer(stddev=0.05)))

    def zero_var(name, shape, dtype):

        return(tf.get_variable(name=name, shape=shape, dtype=dtype, initializer=tf.constant_initializer(0.0)))

    # First Convolutional Layer

    with tf.variable_scope('conv1') as scope:

        # Conv_kernel is 5x5 for all 3 colors and we will create 64 features

        conv1_kernel   = truncated_normal_var(name='conv_kernel1', shape=[5, 5, 3, 64], dtype=tf.float32)

        # We convolve across the image with a stride size of 1

        conv1          = tf.nn.conv2d(input_images, conv1_kernel, [1, 1, 1, 1], padding='SAME')

        # Initialize and add the bias term

        conv1_bias     = zero_var(name='conv_bias1', shape=[64], dtype=tf.float32)

        conv1_add_bias = tf.nn.bias_add(conv1, conv1_bias)

        # ReLU element wise

        relu_conv1     = tf.nn.relu(conv1_add_bias)

    # Max Pooling

    pool1 = tf.nn.max_pool(relu_conv1, ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1],padding='SAME', name='pool_layer1')

    # Local Response Normalization (parameters from paper)

    # paper: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks

    norm1 = tf.nn.lrn(pool1, depth_radius=5, bias=2.0, alpha=1e-3, beta=0.75, name='norm1')

    # Second Convolutional Layer

    with tf.variable_scope('conv2') as scope:

        # Conv kernel is 5x5, across all prior 64 features and we create 64 more features

        conv2_kernel   = truncated_normal_var(name='conv_kernel2', shape=[5, 5, 64, 64], dtype=tf.float32)

        # Convolve filter across prior output with stride size of 1

        conv2          = tf.nn.conv2d(norm1, conv2_kernel, [1, 1, 1, 1], padding='SAME')

        # Initialize and add the bias

        conv2_bias     = zero_var(name='conv_bias2', shape=[64], dtype=tf.float32)

        conv2_add_bias = tf.nn.bias_add(conv2, conv2_bias)

        # ReLU element wise

        relu_conv2     = tf.nn.relu(conv2_add_bias)

    # Max Pooling

    pool2 = tf.nn.max_pool(relu_conv2, ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1], padding='SAME', name='pool_layer2')    

    # Local Response Normalization (parameters from paper)

    norm2 = tf.nn.lrn(pool2, depth_radius=5, bias=2.0, alpha=1e-3, beta=0.75, name='norm2')

    # Reshape output into a single matrix for multiplication for the fully connected layers

    reshaped_output = tf.reshape(norm2, [batch_size, -1])

    reshaped_dim    = reshaped_output.get_shape()[1].value

    # First Fully Connected Layer

    with tf.variable_scope('full1') as scope:

        # Fully connected layer will have 384 outputs.

        full_weight1 = truncated_normal_var(name='full_mult1', shape=[reshaped_dim, 384], dtype=tf.float32)

        full_bias1   = zero_var(name='full_bias1', shape=[384], dtype=tf.float32)

        full_layer1  = tf.nn.relu(tf.add(tf.matmul(reshaped_output, full_weight1), full_bias1))

    # Second Fully Connected Layer

    with tf.variable_scope('full2') as scope:

        # Second fully connected layer has 192 outputs.

        full_weight2 = truncated_normal_var(name='full_mult2', shape=[384, 192], dtype=tf.float32)

        full_bias2   = zero_var(name='full_bias2', shape=[192], dtype=tf.float32)

        full_layer2  = tf.nn.relu(tf.add(tf.matmul(full_layer1, full_weight2), full_bias2))

    # Final Fully Connected Layer -> 10 categories for output (num_targets)

    with tf.variable_scope('full3') as scope:

        # Final fully connected layer has 10 (num_targets) outputs.

        full_weight3 = truncated_normal_var(name='full_mult3', shape=[192, num_targets], dtype=tf.float32)

        full_bias3   =  zero_var(name='full_bias3', shape=[num_targets], dtype=tf.float32)

        final_output = tf.add(tf.matmul(full_layer2, full_weight3), full_bias3)

    return(final_output)

5. Loss and accuracy.

# Loss function

def cifar_loss(logits, targets):

    # Get rid of extra dimensions and cast targets into integers

    targets = tf.squeeze(tf.cast(targets, tf.int32))

    # Calculate cross entropy from logits and targets

    cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(logits, targets)

    # Take the average loss across batch size

    cross_entropy_mean = tf.reduce_mean(cross_entropy, name='cross_entropy')

    return(cross_entropy_mean)

# Accuracy function

def accuracy_of_batch(logits, targets):

    # Make sure targets are integers and drop extra dimensions

    targets = tf.squeeze(tf.cast(targets, tf.int32))

    # Get predicted values by finding which logit is the greatest

    batch_predictions = tf.cast(tf.argmax(logits, 1), tf.int32)

    # Check if they are equal across the batch

    predicted_correctly = tf.equal(batch_predictions, targets)

    # Average the 1's and 0's (True's and False's) across the batch size

    accuracy = tf.reduce_mean(tf.cast(predicted_correctly, tf.float32))

    return(accuracy)

# Train step

def train_step(loss_value, generation_num):

    # Our learning rate is an exponential decay after we wait a fair number of generations

    model_learning_rate = tf.train.exponential_decay(learning_rate, generation_num,

                                                     num_gens_to_wait, lr_decay, staircase=True)

    # Create optimizer

    my_optimizer = tf.train.GradientDescentOptimizer(model_learning_rate)

    # Initialize train step

    train_step = my_optimizer.minimize(loss_value)

    return(train_step)

# Declare loss function

print('Declare Loss Function.')

loss = cifar_loss(model_output, targets)

# Create accuracy function

accuracy = accuracy_of_batch(test_output, test_targets)

# Create training operations

print('Creating the Training Operation.')

generation_num = tf.Variable(0, trainable=False)

train_op = train_step(loss, generation_num)

# Initialize Variables

print('Initializing the Variables.')

init = tf.initialize_all_variables()

sess.run(init)

# Initialize queue (This queue will feed into the model, so no placeholders necessary)

tf.train.start_queue_runners(sess=sess)

6. Training.

# Train CIFAR Model

print('Starting Training')

train_loss = []

test_accuracy = []

for i in range(generations):

    _, loss_value = sess.run([train_op, loss])

    if (i+1) % output_every == 0:

        train_loss.append(loss_value)

        output = 'Generation {}: Loss = {:.5f}'.format((i+1), loss_value)

        print(output)

    if (i+1) % eval_every == 0:

        [temp_accuracy] = sess.run([accuracy])

        test_accuracy.append(temp_accuracy)

        acc_output = ' --- Test Accuracy = {:.2f}%.'.format(100.*temp_accuracy)

        print(acc_output)

# Print loss and accuracy

# Matlotlib code to plot the loss and accuracies

eval_indices   = range(0, generations, eval_every)

output_indices = range(0, generations, output_every)

# Plot loss over time

plt.plot(output_indices, train_loss, 'k-')

plt.title('Softmax Loss per Generation')

plt.xlabel('Generation')

plt.ylabel('Softmax Loss')

plt.show()

# Plot accuracy over time

plt.plot(eval_indices, test_accuracy, 'k-')

plt.title('Test Accuracy')

plt.xlabel('Generation')

plt.ylabel('Accuracy')

plt.show()
# More Advanced CNN Model: CIFAR-10

#---------------------------------------

#

# In this example, we will download the CIFAR-10 images

# and build a CNN model with dropout and regularization

#

# CIFAR is composed ot 50k train and 10k test

# images that are 32x32.

import os

import tarfile

import matplotlib.pyplot as plt

import tensorflow as tf

from six.moves import urllib

from tensorflow.python.framework import ops

ops.reset_default_graph()

# Change Directory

abspath = os.path.abspath(__file__)

dname = os.path.dirname(abspath)

os.chdir(dname)

# Start a graph session

sess = tf.Session()

# Set model parameters

batch_size   = 128

data_dir     = 'temp'

output_every = 50

generations  = 20000

eval_every   = 500

image_height = 32

image_width  = 32

crop_height  = 24

crop_width   = 24

num_channels = 3

num_targets  = 10

extract_folder = 'cifar-10-batches-bin'

# Exponential Learning Rate Decay Params

learning_rate    = 0.1

lr_decay         = 0.1

num_gens_to_wait = 250.

# Extract model parameters

image_vec_length = image_height * image_width * num_channels

record_length    = 1 + image_vec_length # ( + 1 for the 0-9 label)

# Create folder then load data

data_dir = 'temp'

if not os.path.exists(data_dir):

    os.makedirs(data_dir)

cifar10_url = 'http://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz'

# Check if file exists, otherwise download it

data_file = os.path.join(data_dir, 'cifar-10-binary.tar.gz')

if os.path.isfile(data_file):

    pass

else:

    # Download file

    def progress(block_num, block_size, total_size):

        progress_info = [cifar10_url, float(block_num * block_size) / float(total_size) * 100.0]

        print('\r Downloading {} - {:.2f}%'.format(*progress_info), end="")

    filepath, _ = urllib.request.urlretrieve(cifar10_url, data_file, progress)

    # Extract file

    tarfile.open(filepath, 'r:gz').extractall(data_dir)

# Define CIFAR reader

def read_cifar_files(filename_queue, distort_images = True):

    reader = tf.FixedLengthRecordReader(record_bytes=record_length)

    key, record_string = reader.read(filename_queue)

    record_bytes = tf.decode_raw(record_string, tf.uint8)

    image_label  = tf.cast(tf.slice(record_bytes, [0], [1]), tf.int32)

    # Extract image

    image_extracted = tf.reshape(tf.slice(record_bytes, [1], [image_vec_length]),

                                 [num_channels, image_height, image_width])

    # Reshape image

    image_uint8image = tf.transpose(image_extracted, [1, 2, 0])

    reshaped_image   = tf.cast(image_uint8image, tf.float32)

    # Randomly Crop image

    final_image = tf.image.resize_image_with_crop_or_pad(reshaped_image, crop_width, crop_height)

    if distort_images:

        # Randomly flip the image horizontally, change the brightness and contrast

        final_image = tf.image.random_flip_left_right(final_image)

        final_image = tf.image.random_brightness(final_image,max_delta=63)

        final_image = tf.image.random_contrast(final_image,lower=0.2, upper=1.8)

    # Normalize whitening

    final_image = tf.image.per_image_standardization(final_image)

    return(final_image, image_label)

# Create a CIFAR image pipeline from reader

def input_pipeline(batch_size, train_logical=True):

    if train_logical:

        files = [os.path.join(data_dir, extract_folder, 'data_batch_{}.bin'.format(i)) for i in range(1,6)]

    else:

        files = [os.path.join(data_dir, extract_folder, 'test_batch.bin')]

    filename_queue = tf.train.string_input_producer(files)

    image, label   = read_cifar_files(filename_queue)

    # min_after_dequeue defines how big a buffer we will randomly sample

    #   from -- bigger means better shuffling but slower start up and more

    #   memory used.

    # capacity must be larger than min_after_dequeue and the amount larger

    #   determines the maximum we will prefetch.  Recommendation:

    #   min_after_dequeue + (num_threads + a small safety margin) * batch_size

    min_after_dequeue = 5000

    capacity = min_after_dequeue + 3 * batch_size

    example_batch, label_batch = tf.train.shuffle_batch([image, label],

                                                        batch_size=batch_size,

                                                        capacity=capacity,

                                                        min_after_dequeue=min_after_dequeue)

    return(example_batch, label_batch)

# Define the model architecture, this will return logits from images

def cifar_cnn_model(input_images, batch_size, train_logical=True):

    def truncated_normal_var(name, shape, dtype):

        return(tf.get_variable(name=name, shape=shape, dtype=dtype, initializer=tf.truncated_normal_initializer(stddev=0.05)))

    def zero_var(name, shape, dtype):

        return(tf.get_variable(name=name, shape=shape, dtype=dtype, initializer=tf.constant_initializer(0.0)))

    # First Convolutional Layer

    with tf.variable_scope('conv1') as scope:

        # Conv_kernel is 5x5 for all 3 colors and we will create 64 features

        conv1_kernel = truncated_normal_var(name='conv_kernel1', shape=[5, 5, 3, 64], dtype=tf.float32)

        # We convolve across the image with a stride size of 1

        conv1 = tf.nn.conv2d(input_images, conv1_kernel, [1, 1, 1, 1], padding='SAME')

        # Initialize and add the bias term

        conv1_bias = zero_var(name='conv_bias1', shape=[64], dtype=tf.float32)

        conv1_add_bias = tf.nn.bias_add(conv1, conv1_bias)

        # ReLU element wise

        relu_conv1 = tf.nn.relu(conv1_add_bias)

    # Max Pooling

    pool1 = tf.nn.max_pool(relu_conv1, ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1],padding='SAME', name='pool_layer1')

    # Local Response Normalization (parameters from paper)

    # paper: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks

    norm1 = tf.nn.lrn(pool1, depth_radius=5, bias=2.0, alpha=1e-3, beta=0.75, name='norm1')

    # Second Convolutional Layer

    with tf.variable_scope('conv2') as scope:

        # Conv kernel is 5x5, across all prior 64 features and we create 64 more features

        conv2_kernel = truncated_normal_var(name='conv_kernel2', shape=[5, 5, 64, 64], dtype=tf.float32)

        # Convolve filter across prior output with stride size of 1

        conv2 = tf.nn.conv2d(norm1, conv2_kernel, [1, 1, 1, 1], padding='SAME')

        # Initialize and add the bias

        conv2_bias = zero_var(name='conv_bias2', shape=[64], dtype=tf.float32)

        conv2_add_bias = tf.nn.bias_add(conv2, conv2_bias)

        # ReLU element wise

        relu_conv2 = tf.nn.relu(conv2_add_bias)

    # Max Pooling

    pool2 = tf.nn.max_pool(relu_conv2, ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1], padding='SAME', name='pool_layer2')    

     # Local Response Normalization (parameters from paper)

    norm2 = tf.nn.lrn(pool2, depth_radius=5, bias=2.0, alpha=1e-3, beta=0.75, name='norm2')

    # Reshape output into a single matrix for multiplication for the fully connected layers

    reshaped_output = tf.reshape(norm2, [batch_size, -1])

    reshaped_dim = reshaped_output.get_shape()[1].value

    # First Fully Connected Layer

    with tf.variable_scope('full1') as scope:

        # Fully connected layer will have 384 outputs.

        full_weight1 = truncated_normal_var(name='full_mult1', shape=[reshaped_dim, 384], dtype=tf.float32)

        full_bias1 = zero_var(name='full_bias1', shape=[384], dtype=tf.float32)

        full_layer1 = tf.nn.relu(tf.add(tf.matmul(reshaped_output, full_weight1), full_bias1))

    # Second Fully Connected Layer

    with tf.variable_scope('full2') as scope:

        # Second fully connected layer has 192 outputs.

        full_weight2 = truncated_normal_var(name='full_mult2', shape=[384, 192], dtype=tf.float32)

        full_bias2 = zero_var(name='full_bias2', shape=[192], dtype=tf.float32)

        full_layer2 = tf.nn.relu(tf.add(tf.matmul(full_layer1, full_weight2), full_bias2))

    # Final Fully Connected Layer -> 10 categories for output (num_targets)

    with tf.variable_scope('full3') as scope:

        # Final fully connected layer has 10 (num_targets) outputs.

        full_weight3 = truncated_normal_var(name='full_mult3', shape=[192, num_targets], dtype=tf.float32)

        full_bias3 =  zero_var(name='full_bias3', shape=[num_targets], dtype=tf.float32)

        final_output = tf.add(tf.matmul(full_layer2, full_weight3), full_bias3)

    return(final_output)

# Loss function

def cifar_loss(logits, targets):

    # Get rid of extra dimensions and cast targets into integers

    targets = tf.squeeze(tf.cast(targets, tf.int32))

    # Calculate cross entropy from logits and targets

    cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(logits, targets)

    # Take the average loss across batch size

    cross_entropy_mean = tf.reduce_mean(cross_entropy, name='cross_entropy')

    return(cross_entropy_mean)

# Train step

def train_step(loss_value, generation_num):

    # Our learning rate is an exponential decay after we wait a fair number of generations

    model_learning_rate = tf.train.exponential_decay(learning_rate, generation_num,

                                                     num_gens_to_wait, lr_decay, staircase=True)

    # Create optimizer

    my_optimizer = tf.train.GradientDescentOptimizer(model_learning_rate)

    # Initialize train step

    train_step = my_optimizer.minimize(loss_value)

    return(train_step)

# Accuracy function

def accuracy_of_batch(logits, targets):

    # Make sure targets are integers and drop extra dimensions

    targets = tf.squeeze(tf.cast(targets, tf.int32))

    # Get predicted values by finding which logit is the greatest

    batch_predictions = tf.cast(tf.argmax(logits, 1), tf.int32)

    # Check if they are equal across the batch

    predicted_correctly = tf.equal(batch_predictions, targets)

    # Average the 1's and 0's (True's and False's) across the batch size

    accuracy = tf.reduce_mean(tf.cast(predicted_correctly, tf.float32))

    return(accuracy)

# Get data

print('Getting/Transforming Data.')

# Initialize the data pipeline

images, targets = input_pipeline(batch_size, train_logical=True)

# Get batch test images and targets from pipline

test_images, test_targets = input_pipeline(batch_size, train_logical=False)

# Declare Model

print('Creating the CIFAR10 Model.')

with tf.variable_scope('model_definition') as scope:

    # Declare the training network model

    model_output = cifar_cnn_model(images, batch_size)

    # This is very important!!!  We must set the scope to REUSE the variables,

    #  otherwise, when we set the test network model, it will create new random

    #  variables.  Otherwise we get random evaluations on the test batches.

    scope.reuse_variables()

    test_output = cifar_cnn_model(test_images, batch_size)

# Declare loss function

print('Declare Loss Function.')

loss = cifar_loss(model_output, targets)

# Create accuracy function

accuracy = accuracy_of_batch(test_output, test_targets)

# Create training operations

print('Creating the Training Operation.')

generation_num = tf.Variable(0, trainable=False)

train_op = train_step(loss, generation_num)

# Initialize Variables

print('Initializing the Variables.')

init = tf.initialize_all_variables()

sess.run(init)

# Initialize queue (This queue will feed into the model, so no placeholders necessary)

tf.train.start_queue_runners(sess=sess)

# Train CIFAR Model

print('Starting Training')

train_loss = []

test_accuracy = []

for i in range(generations):

    _, loss_value = sess.run([train_op, loss])

    if (i+1) % output_every == 0:

        train_loss.append(loss_value)

        output = 'Generation {}: Loss = {:.5f}'.format((i+1), loss_value)

        print(output)

    if (i+1) % eval_every == 0:

        [temp_accuracy] = sess.run([accuracy])

        test_accuracy.append(temp_accuracy)

        acc_output = ' --- Test Accuracy = {:.2f}%.'.format(100.*temp_accuracy)

        print(acc_output)

# Print loss and accuracy

# Matlotlib code to plot the loss and accuracies

eval_indices = range(0, generations, eval_every)

output_indices = range(0, generations, output_every)

# Plot loss over time

plt.plot(output_indices, train_loss, 'k-')

plt.title('Softmax Loss per Generation')

plt.xlabel('Generation')

plt.ylabel('Softmax Loss')

plt.show()

# Plot accuracy over time

plt.plot(eval_indices, test_accuracy, 'k-')

plt.title('Test Accuracy')

plt.xlabel('Generation')

plt.ylabel('Accuracy')

plt.show()

完整代码

# student @ user-ubuntu in ~/work/jeff/cnn_cifar [3:43:05]

$ python cnn_cifar10.py

2017-07-27 03:43:18.521404: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.

2017-07-27 03:43:18.521432: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.

2017-07-27 03:43:18.521439: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.

2017-07-27 03:43:18.521461: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.

2017-07-27 03:43:18.521482: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.

2017-07-27 03:43:18.971638: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties:

name: GeForce GTX 1080 Ti

major: 6 minor: 1 memoryClockRate (GHz) 1.582

pciBusID 0000:82:00.0

Total memory: 10.91GiB

Free memory: 10.75GiB

2017-07-27 03:43:19.404592: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x2b7d1e0 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.

2017-07-27 03:43:19.406002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 1 with properties:

name: GeForce GTX 1080 Ti

major: 6 minor: 1 memoryClockRate (GHz) 1.582

pciBusID 0000:83:00.0

Total memory: 10.91GiB

Free memory: 10.75GiB

2017-07-27 03:43:19.407105: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0 1

2017-07-27 03:43:19.407122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   Y Y

2017-07-27 03:43:19.407127: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 1:   Y Y

2017-07-27 03:43:19.407143: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:82:00.0)

2017-07-27 03:43:19.407150: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:83:00.0)

Getting/Transforming Data.

Creating the CIFAR10 Model.

Declare Loss Function.

Creating the Training Operation.

Initializing the Variables.

WARNING:tensorflow:From /home/sean/virtualenv/tensorflow-py3.5/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py:170: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.

Instructions for updating:

Use `tf.global_variables_initializer` instead.

Starting Training

Generation 50 : Loss = 1.98874

Generation 100: Loss = 1.89331

Generation 150: Loss = 2.14434

Generation 200: Loss = 1.62269

Generation 250: Loss = 1.73568

Generation 300: Loss = 1.67207

Generation 350: Loss = 1.47158

Generation 400: Loss = 1.70603

Generation 450: Loss = 1.70473

Generation 500: Loss = 1.54680

 --- Test Accuracy = 50.00%.

Generation 550: Loss = 1.47264

Generation 600: Loss = 1.33211

Generation 650: Loss = 1.28465

Generation 700: Loss = 1.47795

Generation 750: Loss = 1.54336

Generation 800: Loss = 1.32847

Generation 850: Loss = 1.30343

Generation 900: Loss = 1.21732

Generation 950: Loss = 1.38395

Generation 1000: Loss = 1.18652

 --- Test Accuracy = 58.59%.

Generation 1050: Loss = 1.18199

Generation 1100: Loss = 1.26724

Generation 1150: Loss = 1.18451

Generation 1200: Loss = 1.38382

Generation 1250: Loss = 1.10484

Generation 1300: Loss = 1.22248

Generation 1350: Loss = 1.03363

Generation 1400: Loss = 1.19343

Generation 1450: Loss = 1.21833

Generation 1500: Loss = 1.28556

 --- Test Accuracy = 64.06%.

Generation 1550: Loss = 1.05083

Generation 1600: Loss = 1.12365

Generation 1650: Loss = 0.98373

Generation 1700: Loss = 0.98653

Generation 1750: Loss = 0.89979

Generation 1800: Loss = 1.15915

Generation 1850: Loss = 0.97605

Generation 1900: Loss = 0.98232

Generation 1950: Loss = 0.93118

Generation 2000: Loss = 0.84955

 --- Test Accuracy = 71.88%.

Generation 2050: Loss = 1.01471

Generation 2100: Loss = 1.03468

Generation 2150: Loss = 1.03076

Generation 2200: Loss = 1.12238

Generation 2250: Loss = 0.72818

Generation 2300: Loss = 0.92140

Generation 2350: Loss = 0.96073

Generation 2400: Loss = 0.90507

Generation 2450: Loss = 0.84059

Generation 2500: Loss = 1.00707

 --- Test Accuracy = 64.84%.

Generation 2550: Loss = 0.77818

Generation 2600: Loss = 0.91769

Generation 2650: Loss = 0.83464

Generation 2700: Loss = 0.82919

Generation 2750: Loss = 0.71690

Generation 2800: Loss = 0.69588

Generation 2850: Loss = 0.81282

Generation 2900: Loss = 0.86472

Generation 2950: Loss = 0.68893

Generation 3000: Loss = 0.68487

 --- Test Accuracy = 66.41%.

Generation 3050: Loss = 0.77830

Generation 3100: Loss = 0.98387

Generation 3150: Loss = 0.67766

Generation 3200: Loss = 0.83822

Generation 3250: Loss = 0.72022

Generation 3300: Loss = 0.61615

Generation 3350: Loss = 0.80152

Generation 3400: Loss = 0.61803

Generation 3450: Loss = 0.69104

Generation 3500: Loss = 0.86474

 --- Test Accuracy = 71.09%.

Generation 3550: Loss = 0.56136

Generation 3600: Loss = 0.83764

Generation 3650: Loss = 0.75091

Generation 3700: Loss = 0.57823

Generation 3750: Loss = 0.52850

Generation 3800: Loss = 0.53191

Generation 3850: Loss = 0.65577

Generation 3900: Loss = 0.70614

Generation 3950: Loss = 0.57539

Generation 4000: Loss = 0.61946

 --- Test Accuracy = 71.09%.

Generation 4050: Loss = 0.50912

Generation 4100: Loss = 0.59709

Generation 4150: Loss = 0.63275

Generation 4200: Loss = 0.73160

Generation 4250: Loss = 0.69023

Generation 4300: Loss = 0.68340

Generation 4350: Loss = 0.54651

Generation 4400: Loss = 0.66809

Generation 4450: Loss = 0.54778

Generation 4500: Loss = 0.49987

 --- Test Accuracy = 66.41%.

Generation 4550: Loss = 0.52780

Generation 4600: Loss = 0.47527

Generation 4650: Loss = 0.56457

Generation 4700: Loss = 0.49000

Generation 4750: Loss = 0.62392

Generation 4800: Loss = 0.53709

Generation 4850: Loss = 0.46020

Generation 4900: Loss = 0.58521

Generation 4950: Loss = 0.52085

Generation 5000: Loss = 0.56563

 --- Test Accuracy = 76.56%.

Generation 5050: Loss = 0.62101

Generation 5100: Loss = 0.68806

Generation 5150: Loss = 0.56646

Generation 5200: Loss = 0.55054

Generation 5250: Loss = 0.70789

Generation 5300: Loss = 0.48409

Generation 5350: Loss = 0.48703

Generation 5400: Loss = 0.53602

Generation 5450: Loss = 0.53750

Generation 5500: Loss = 0.44592

 --- Test Accuracy = 71.88%.

Generation 5550: Loss = 0.61261

Generation 5600: Loss = 0.57290

Generation 5650: Loss = 0.52776

Generation 5700: Loss = 0.49262

Generation 5750: Loss = 0.44058

Generation 5800: Loss = 0.62443

Generation 5850: Loss = 0.38249

Generation 5900: Loss = 0.39162

Generation 5950: Loss = 0.49900

Generation 6000: Loss = 0.60641

 --- Test Accuracy = 73.44%.

Generation 6050: Loss = 0.52156

Generation 6100: Loss = 0.50984

Generation 6150: Loss = 0.62414

Generation 6200: Loss = 0.56085

Generation 6250: Loss = 0.45930

Generation 6300: Loss = 0.41330

Generation 6350: Loss = 0.46615

Generation 6400: Loss = 0.48824

Generation 6450: Loss = 0.61569

Generation 6500: Loss = 0.54841

 --- Test Accuracy = 72.66%.

Generation 6550: Loss = 0.45108

Generation 6600: Loss = 0.36974

Generation 6650: Loss = 0.42269

Generation 6700: Loss = 0.31257

Generation 6750: Loss = 0.39991

Generation 6800: Loss = 0.34907

Generation 6850: Loss = 0.34459

Generation 6900: Loss = 0.39457

Generation 6950: Loss = 0.29138

Generation 7000: Loss = 0.40070

 --- Test Accuracy = 71.09%.

Generation 7050: Loss = 0.31275

Generation 7100: Loss = 0.37386

Generation 7150: Loss = 0.57231

Generation 7200: Loss = 0.33384

Generation 7250: Loss = 0.39317

Generation 7300: Loss = 0.27306

Generation 7350: Loss = 0.42451

Generation 7400: Loss = 0.44812

Generation 7450: Loss = 0.40212

Generation 7500: Loss = 0.33456

 --- Test Accuracy = 71.88%.

Generation 7550: Loss = 0.33886

Generation 7600: Loss = 0.35627

Generation 7650: Loss = 0.37291

Generation 7700: Loss = 0.36350

Generation 7750: Loss = 0.61906

Generation 7800: Loss = 0.44072

Generation 7850: Loss = 0.53827

Generation 7900: Loss = 0.40603

Generation 7950: Loss = 0.34712

Generation 8000: Loss = 0.37340

 --- Test Accuracy = 74.22%.

Generation 8050: Loss = 0.34767

Generation 8100: Loss = 0.27752

Generation 8150: Loss = 0.39700

Generation 8200: Loss = 0.32144

Generation 8250: Loss = 0.29297

Generation 8300: Loss = 0.25248

Generation 8350: Loss = 0.31232

Generation 8400: Loss = 0.37384

Generation 8450: Loss = 0.19988

Generation 8500: Loss = 0.35983

 --- Test Accuracy = 70.31%.

Generation 8550: Loss = 0.24563

Generation 8600: Loss = 0.26254

Generation 8650: Loss = 0.28202

Generation 8700: Loss = 0.38843

Generation 8750: Loss = 0.36233

Generation 8800: Loss = 0.18249

Generation 8850: Loss = 0.28049

Generation 8900: Loss = 0.21987

Generation 8950: Loss = 0.27884

Generation 9000: Loss = 0.26830

 --- Test Accuracy = 78.91%.

Generation 9050: Loss = 0.21410

Generation 9100: Loss = 0.18955

Generation 9150: Loss = 0.21886

Generation 9200: Loss = 0.35513

Generation 9250: Loss = 0.23994

Generation 9300: Loss = 0.30040

Generation 9350: Loss = 0.27230

Generation 9400: Loss = 0.24417

Generation 9450: Loss = 0.23737

Generation 9500: Loss = 0.24323

 --- Test Accuracy = 71.88%.

Generation 9550: Loss = 0.28200

Generation 9600: Loss = 0.37996

Generation 9650: Loss = 0.14036

Generation 9700: Loss = 0.25095

Generation 9750: Loss = 0.24847

Generation 9800: Loss = 0.31754

Generation 9850: Loss = 0.17151

Generation 9900: Loss = 0.19960

Generation 9950: Loss = 0.24201

Generation 10000: Loss = 0.25191

 --- Test Accuracy = 71.88%.

Generation 10050: Loss = 0.21244

Generation 10100: Loss = 0.31911

Generation 10150: Loss = 0.25067

Generation 10200: Loss = 0.17353

Generation 10250: Loss = 0.17035

Generation 10300: Loss = 0.20111

Generation 10350: Loss = 0.24000

Generation 10400: Loss = 0.28682

Generation 10450: Loss = 0.27803

Generation 10500: Loss = 0.22228

 --- Test Accuracy = 74.22%.

Generation 10550: Loss = 0.20168

Generation 10600: Loss = 0.18150

Generation 10650: Loss = 0.12649

Generation 10700: Loss = 0.21024

Generation 10750: Loss = 0.13210

Generation 10800: Loss = 0.21463

Generation 10850: Loss = 0.19228

Generation 10900: Loss = 0.20855

Generation 10950: Loss = 0.09159

Generation 11000: Loss = 0.19000

 --- Test Accuracy = 71.09%.

Generation 11050: Loss = 0.16792

Generation 11100: Loss = 0.18264

Generation 11150: Loss = 0.20756

Generation 11200: Loss = 0.23574

Generation 11250: Loss = 0.25095

Generation 11300: Loss = 0.19270

Generation 11350: Loss = 0.19303

Generation 11400: Loss = 0.16534

Generation 11450: Loss = 0.29888

Generation 11500: Loss = 0.17793

 --- Test Accuracy = 78.91%.

Generation 11550: Loss = 0.15598

Generation 11600: Loss = 0.12160

Generation 11650: Loss = 0.26322

Generation 11700: Loss = 0.10899

Generation 11750: Loss = 0.11561

Generation 11800: Loss = 0.16404

Generation 11850: Loss = 0.18666

Generation 11900: Loss = 0.15152

Generation 11950: Loss = 0.22033

Generation 12000: Loss = 0.17022

 --- Test Accuracy = 77.34%.

Generation 12050: Loss = 0.06982

Generation 12100: Loss = 0.11614

Generation 12150: Loss = 0.22383

Generation 12200: Loss = 0.14770

Generation 12250: Loss = 0.12691

Generation 12300: Loss = 0.13115

Generation 12350: Loss = 0.15366

Generation 12400: Loss = 0.10993

Generation 12450: Loss = 0.12453

Generation 12500: Loss = 0.11822

 --- Test Accuracy = 68.75%.

Generation 12550: Loss = 0.08440

Generation 12600: Loss = 0.10500

Generation 12650: Loss = 0.09079

Generation 12700: Loss = 0.17050

Generation 12750: Loss = 0.16910

Generation 12800: Loss = 0.16500

Generation 12850: Loss = 0.10901

Generation 12900: Loss = 0.06830

Generation 12950: Loss = 0.06736

Generation 13000: Loss = 0.15788

 --- Test Accuracy = 75.00%.

Generation 13050: Loss = 0.13596

Generation 13100: Loss = 0.11368

Generation 13150: Loss = 0.15130

Generation 13200: Loss = 0.16115

Generation 13250: Loss = 0.08005

Generation 13300: Loss = 0.37412

Generation 13350: Loss = 0.08087

Generation 13400: Loss = 0.05354

Generation 13450: Loss = 0.14977

Generation 13500: Loss = 0.06454

 --- Test Accuracy = 76.56%.

Generation 13550: Loss = 0.10611

Generation 13600: Loss = 0.14358

Generation 13650: Loss = 0.30438

Generation 13700: Loss = 0.12326

Generation 13750: Loss = 0.12546

Generation 13800: Loss = 0.05507

Generation 13850: Loss = 0.10522

Generation 13900: Loss = 0.14672

Generation 13950: Loss = 0.08316

Generation 14000: Loss = 0.04716

 --- Test Accuracy = 81.25%.

Generation 14050: Loss = 0.05569

Generation 14100: Loss = 0.07380

Generation 14150: Loss = 0.08503

Generation 14200: Loss = 0.09180

Generation 14250: Loss = 0.12643

Generation 14300: Loss = 0.28318

Generation 14350: Loss = 0.10349

Generation 14400: Loss = 0.08242

Generation 14450: Loss = 0.14991

Generation 14500: Loss = 0.09186

 --- Test Accuracy = 68.75%.

Generation 14550: Loss = 0.05001

Generation 14600: Loss = 0.09397

Generation 14650: Loss = 0.15350

Generation 14700: Loss = 0.05397

Generation 14750: Loss = 0.07973

Generation 14800: Loss = 0.04911

Generation 14850: Loss = 0.14983

Generation 14900: Loss = 0.04087

Generation 14950: Loss = 0.09777

Generation 15000: Loss = 0.06766

 --- Test Accuracy = 79.69%.

Generation 15050: Loss = 0.08833

Generation 15100: Loss = 0.05564

Generation 15150: Loss = 0.14910

Generation 15200: Loss = 0.09593

Generation 15250: Loss = 0.02290

Generation 15300: Loss = 0.09136

Generation 15350: Loss = 0.05094

Generation 15400: Loss = 0.06671

Generation 15450: Loss = 0.06932

Generation 15500: Loss = 0.05988

 --- Test Accuracy = 71.09%.

Generation 15550: Loss = 0.03620

Generation 15600: Loss = 0.11110

Generation 15650: Loss = 0.09207

Generation 15700: Loss = 0.12269

Generation 15750: Loss = 0.07687

Generation 15800: Loss = 0.13080

Generation 15850: Loss = 0.06997

Generation 15900: Loss = 0.08472

Generation 15950: Loss = 0.09175

Generation 16000: Loss = 0.10958

 --- Test Accuracy = 74.22%.

Generation 16050: Loss = 0.06961

Generation 16100: Loss = 0.04545

Generation 16150: Loss = 0.08067

Generation 16200: Loss = 0.09693

Generation 16250: Loss = 0.10157

Generation 16300: Loss = 0.12212

Generation 16350: Loss = 0.09758

Generation 16400: Loss = 0.11395

Generation 16450: Loss = 0.06691

Generation 16500: Loss = 0.07225

 --- Test Accuracy = 78.12%.

Generation 16550: Loss = 0.06004

Generation 16600: Loss = 0.06961

Generation 16650: Loss = 0.18485

Generation 16700: Loss = 0.08379

Generation 16750: Loss = 0.06583

Generation 16800: Loss = 0.07328

Generation 16850: Loss = 0.06474

Generation 16900: Loss = 0.09860

Generation 16950: Loss = 0.05844

Generation 17000: Loss = 0.05786

 --- Test Accuracy = 72.66%.

Generation 17050: Loss = 0.07321

Generation 17100: Loss = 0.12127

Generation 17150: Loss = 0.05446

Generation 17200: Loss = 0.03569

Generation 17250: Loss = 0.05022

Generation 17300: Loss = 0.11921

Generation 17350: Loss = 0.04017

Generation 17400: Loss = 0.04406

Generation 17450: Loss = 0.06000

Generation 17500: Loss = 0.11646

 --- Test Accuracy = 71.88%.

Generation 17550: Loss = 0.14290

Generation 17600: Loss = 0.04812

Generation 17650: Loss = 0.06327

Generation 17700: Loss = 0.13356

Generation 17750: Loss = 0.09151

Generation 17800: Loss = 0.13200

Generation 17850: Loss = 0.16656

Generation 17900: Loss = 0.05319

Generation 17950: Loss = 0.07239

Generation 18000: Loss = 0.07355

 --- Test Accuracy = 75.78%.

Generation 18050: Loss = 0.09703

Generation 18100: Loss = 0.02250

Generation 18150: Loss = 0.02004

Generation 18200: Loss = 0.12326

Generation 18250: Loss = 0.15379

Generation 18300: Loss = 0.03012

Generation 18350: Loss = 0.05909

Generation 18400: Loss = 0.05413

Generation 18450: Loss = 0.06055

Generation 18500: Loss = 0.04617

 --- Test Accuracy = 75.00%.

Generation 18550: Loss = 0.11713

Generation 18600: Loss = 0.10103

Generation 18650: Loss = 0.08530

Generation 18700: Loss = 0.07523

Generation 18750: Loss = 0.06605

Generation 18800: Loss = 0.04995

Generation 18850: Loss = 0.04141

Generation 18900: Loss = 0.04708

Generation 18950: Loss = 0.04045

Generation 19000: Loss = 0.03030

 --- Test Accuracy = 78.91%.

Generation 19050: Loss = 0.02257

Generation 19100: Loss = 0.01894

Generation 19150: Loss = 0.06192

Generation 19200: Loss = 0.15686

Generation 19250: Loss = 0.03990

Generation 19300: Loss = 0.07178

Generation 19350: Loss = 0.07857

Generation 19400: Loss = 0.06567

Generation 19450: Loss = 0.04735

Generation 19500: Loss = 0.12532

 --- Test Accuracy = 67.19%.

Generation 19550: Loss = 0.02739

Generation 19600: Loss = 0.04494

Generation 19650: Loss = 0.16667

Generation 19700: Loss = 0.08560

Generation 19750: Loss = 0.11396

Generation 19800: Loss = 0.08600

Generation 19850: Loss = 0.04694

Generation 19900: Loss = 0.06692

Generation 19950: Loss = 0.03973

Generation 20000: Loss = 0.01563

 --- Test Accuracy = 70.31%.

(tensorflow-py3.5)

# student @ user-ubuntu in ~/work/jeff/cnn_cifar [4:24:35]

完整日志

[Tensorflow] Cookbook - Object Classification based on CIFAR-10的更多相关文章

  1. 【翻译】TensorFlow卷积神经网络识别CIFAR 10Convolutional Neural Network (CNN)| CIFAR 10 TensorFlow

    原网址:https://data-flair.training/blogs/cnn-tensorflow-cifar-10/ by DataFlair Team · Published May 21, ...

  2. 机器学习-TensorFlow应用之classification和ROC curve

    概述 前面几节讲的是linear regression的内容,这里咱们再讲一个非常常用的一种模型那就是classification,classification顾名思义就是分类的意思,在实际的情况是非 ...

  3. A Novel Multi-label Classification Based on PCA and ML-KNN

     ICIC Express Letters                  ICIC International ⓒ2010 ISSN 1881-803X Volume4, Number5, O ...

  4. 论文笔记：Visual Object Tracking based on Adaptive Siamese and Motion Estimation Network

    Visual Object Tracking based on Adaptive Siamese and Motion Estimation 本文提出一种利用上一帧目标位置坐标,在本帧中找出目标可能出 ...

  5. [Tensorflow] Cookbook - The Tensorflow Way

    本章介绍tf基础知识,主要包括cookbook的第一.二章节. 方针:先会用,后定制 Ref: TensorFlow 如何入门? Ref: 如何高效的学习 TensorFlow 代码? 顺便推荐该领域 ...

  6. tensorflow，object，detection，在model zoom，新下载的模型，WARNING:root:Variable [resnet_v1_50/block1/unit_3/bottleneck_v1/conv3/BatchNorm/gamma] is not available in checkpoint

    现象: WARNING:root:Variable [resnet_v1_50/block1/unit_1/bottleneck_v1/conv1/BatchNorm/beta] is not ava ...

  7. [Tensorflow] Cookbook - Retraining Existing CNNs models - Inception Model

    From: https://github.com/jcjohnson/cnn-benchmarks#alexnet 先大概了解模型,再看如果加载pre-training weight. 关于retai ...

  8. [Tensorflow] Cookbook - Neural Network

    In this chapter, we'll cover the following recipes: Implementing Operational Gates Working with Gate ...

  9. [Tensorflow] Cookbook - CNN

    Convolutional Neural Networks (CNNs) are responsible for the major breakthroughs in image recognitio ...

随机推荐

  1. 使用Quartz搭建定时任务脚手架

    定时任务的实现有很多种,在Spring项目中使用一个注解 @Scheduled就可以很快搞定. 但是很难去管理项目中的定时任务,比如说:系统中有多少定时任务,每个定时任务执行规则,修改执行规则,暂停任 ...

  2. spring cloud: 使用consul来替换config server

    上一篇提到了,eureka 2.x官方停止更新后,可以用consul来替代,如果采用consul的话,其实config server也没必要继续使用了,consul自带kv存储,完全可以取代confi ...

  3. centos7 apache设置伪静态 开启rewrite_module

    设置伪静态除了要生成.htaccess文件外,还需要查看服务器是否开启了rewrite_module.经过一番的纠结,处理方法如下: 编辑Apache配置文件 nano /etc/httpd/conf ...

  4. .ipynb文件转.py文件

    到要转的demo.ipynb所在目录下,然后输入命令: jupyter nbconvert --to script demo.ipynb

  5. useradd 命令的常见用法

    在Linux系统中 useradd 是个很基本的命令,但是使用起来却很不直观.以至于在 Ubuntu 中居然添加了一个 adduser 命令来简化添加用户的操作.本文主要描述笔者在学习使用 usera ...

  6. (98)Address already in use: make_sock: could not bind to address 80 [resolved] (2012-10-11 09:04)

    以前遇到一个问题: sudo /etc/init.d/apache2 start * Starting web server apache2 apache2: Could not reliably d ...

  7. QueryRunner实战(query_update)、BeanList\BeanHandler、MapList\MapHandler、ScalarHandler

    1.c3p0-config.xml <?xml version="1.0" encoding="UTF-8"?> <c3p0-config&g ...

  8. 关于 TVM

    偶然间对 arm 中 mali 显示核心感兴趣,找到的 TVM.将了解到的信息做个备忘. TVM 是 Tensor Virtual Machine 的所写? 官网上,TVM 定义自己为一种 Inter ...

  9. 按装 autoconf 出现 需要安装m4-1.4.16.tar.gz的错误

    root@suzgc # /usr/local/php/bin/phpize ./configure --with-php-config=/usr/local/php/bin/php-config - ...

  10. Mongodb 笔记 - 性能及Java代码

    性能 以下数据都是在千兆网络下测试的结果 写入 数据量的增大会导致内存占满, 因为mongodb会将数据尽可能地载入内存, 索引占用的空间也很可观非安全模式下, 速度取决于内存是否占满能差一个数量级, ...