InceptionV4
目录
在下面的结构图中，每一个inception模块中都有一个1∗1的没有激活层的卷积层，用来扩展通道数，从而补偿因为inception模块导致的维度约间。其中Inception-ResNet-V1的结果与Inception v3相当；Inception-ResNet-V1与Inception v4结果差不多，不过实际过程中Inception v4会明显慢于Inception-ResNet-v2，这也许是因为层数太多了。且在Inception-ResNet结构中，只在传统层的上面使用BN层，而不在合并层上使用BN，虽然处处使用BN是有好处，不过更希望能够将一个完整的组件放入单独的GPU中。因为具有大量激活单元的层会占用过多的显存，所以希望这些地方丢弃BN，从而总体增加Inception模块的数量。使得不需要去解决计算资源和模块什么的权衡问题。
1. inception v4
**图1.1 inception v4 网络结构图**
**图1.2 图1.1的stem和Inception-A部分结构图**
**图1.3 图1.1的Reduction-A和Inception-B部分结构图**
**图1.4 图1.1的Reduction-B和Inception-C部分结构图**
2. Inception-resnet-v1 & Inception-resnet-v2
**图2.1 Inception-resnet-v1 & Inception-resnet-v2的结构图**
2.1 Inception-resnet-v1的组成模块
**图2.1.1 图2.1的stem和Inception-ResNet-A部分结构图**
**图2.1.2 图2.1的Reduction-A和Inception-ResNet-B部分结构图**
**图2.1.3 图2.1的Reduction-B和Inception-ResNet-C部分结构图**
2.2 Inception-resnet-v2的组成模块
**图2.2.1 图2.1的stem和Inception-ResNet-A部分结构图**
**图2.2.2 图2.1的Reduction-A和Inception-ResNet-B部分结构图**
**图2.2.3 图2.1的Reduction-B和Inception-ResNet-C部分结构图**
3. 模型训练
在上述的Inception V4，Inception-Resnet-V1，Inception-ResNet-v2这三个模型中都用到了Reduction-A，他们各自的具体参数如下：
图3.1 不同模型下Reduction-A的模型超参数
作者们在训练的过程中发现，如果通道数超过1000，那么Inception-resnet等网络都会开始变得不稳定，并且过早的就“死掉了”，即在迭代几万次之后，平均池化的前面一层就会生成很多的0值。作者们通过调低学习率，增加BN都没有任何改善。
不过他们发现如果在将残差汇入之前，对残差进行缩小，可以让模型稳定训练，值通常选择[0,1.0.3]，如图3.2
**图3.2 对inception-resnet模块进行最后输出值的等比例缩小**
同样的在ResNet-v1中，何凯明等人也在cifar-10中发现了模型的不稳定现象：即在特别深的网络基础上去训cifar-10，需要先以0.01的学习率去训练，然后在以0.1的学习率训练。
不过这里的作者们认为如果通道数特别多的话，即使以特别低的学习率（0.00001）训练也无法让模型收敛，如果之后再用大学习率，那么就会轻松的破坏掉之前的成果。然而简单的缩小残差的输出值有助于学习的稳定，即使进行了简单的缩小，那么对最终结果也造成不了多大的损失，反而有助于稳定训练。
- 在inception-resnet-v1与inception v3的对比中，inception-resnet-v1虽然训练速度更快，不过最后结果有那么一丢丢的差于inception v3；
- 而在inception-resnet-v2与inception v4的对比中，inception-resnet-v2的训练速度更块，而且结果比inception v4也更好一点。所以最后胜出的就是inception-resnet-v2。
**图3.3 不同模型的结果对比**
4. 代码
4.1 Inception-V4
from keras.layers import Input
from keras.layers.merge import concatenate
from keras.layers import Dense, Dropout, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras import backend as K
from keras.models import Model
from keras.utils import plot_model
CONV_BLOCK_COUNT = 0 # 用来命名计数卷积编号
INCEPTION_A_COUNT = 0
INCEPTION_B_COUNT = 0
INCEPTION_C_COUNT = 0
def conv_block(x, nb_filters, nb_row, nb_col, strides=(1, 1), padding='same', use_bias=False):
global CONV_BLOCK_COUNT
CONV_BLOCK_COUNT += 1
with K.name_scope('conv_block_'+str(CONV_BLOCK_COUNT)):
x = Conv2D(filters=nb_filters,
kernel_size=(nb_row, nb_col),
strides=strides,
padding=padding,
use_bias=use_bias)(x)
x = BatchNormalization(axis=-1, momentum=0.9997, scale=False)(x)
x = Activation("relu")(x)
return x
def stem(x_input):
with K.name_scope('stem'):
x = conv_block(x_input, 32, 3, 3, strides=(2, 2), padding='valid')
x = conv_block(x, 32, 3, 3, padding='valid')
x = conv_block(x, 64, 3, 3)
x1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x)
x2 = conv_block(x, 96, 3, 3, strides=(2, 2), padding='valid')
x = concatenate([x1, x2], axis=-1)
x1 = conv_block(x, 64, 1, 1)
x1 = conv_block(x1, 96, 3, 3, padding='valid')
x2 = conv_block(x, 64, 1, 1)
x2 = conv_block(x2, 64, 7, 1)
x2 = conv_block(x2, 64, 1, 7)
x2 = conv_block(x2, 96, 3, 3, padding='valid')
x = concatenate([x1, x2], axis=-1)
x1 = conv_block(x, 192, 3, 3, strides=(2, 2), padding='valid')
x2 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x)
merged_vector = concatenate([x1, x2], axis=-1)
return merged_vector
def inception_A(x_input):
"""35*35 卷积块"""
global INCEPTION_A_COUNT
INCEPTION_A_COUNT += 1
with K.name_scope('inception_A' + str(INCEPTION_A_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input) # 35 * 35 * 192
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 96, 1, 1) # 35 * 35 * 96
conv1x1 = conv_block(x_input, 96, 1, 1) # 35 * 35 * 96
conv1x1_3x3 = conv_block(x_input, 64, 1, 1) # 35 * 35 * 64
conv1x1_3x3 = conv_block(conv1x1_3x3, 96, 3, 3) # 35 * 35 * 96
conv3x3_3x3 = conv_block(x_input, 64, 1, 1) # 35 * 35 * 64
conv3x3_3x3 = conv_block(conv3x3_3x3, 96, 3, 3) # 35 * 35 * 96
conv3x3_3x3 = conv_block(conv3x3_3x3, 96, 3, 3) # 35 * 35 * 96
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv1x1_3x3, conv3x3_3x3], axis=-1) # 35 * 35 * 384
return merged_vector
def inception_B(x_input):
"""17*17 卷积块"""
global INCEPTION_B_COUNT
INCEPTION_B_COUNT += 1
with K.name_scope('inception_B' + str(INCEPTION_B_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input)
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 128, 1, 1)
conv1x1 = conv_block(x_input, 384, 1, 1)
conv1x7_1x7 = conv_block(x_input, 192, 1, 1)
conv1x7_1x7 = conv_block(conv1x7_1x7, 224, 1, 7)
conv1x7_1x7 = conv_block(conv1x7_1x7, 256, 1, 7)
conv2_1x7_7x1 = conv_block(x_input, 192, 1, 1)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 192, 1, 7)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 224, 7, 1)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 224, 1, 7)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 256, 7, 1)
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv1x7_1x7, conv2_1x7_7x1], axis=-1)
return merged_vector
def inception_C(x_input):
"""8*8 卷积块"""
global INCEPTION_C_COUNT
INCEPTION_C_COUNT += 1
with K.name_scope('Inception_C' + str(INCEPTION_C_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input)
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 256, 1, 1)
conv1x1 = conv_block(x_input, 256, 1, 1)
# 用 1x3 和 3x1 替代 3x3
conv3x3_1x1 = conv_block(x_input, 384, 1, 1)
conv3x3_1 = conv_block(conv3x3_1x1, 256, 1, 3)
conv3x3_2 = conv_block(conv3x3_1x1, 256, 3, 1)
conv2_3x3_1x1 = conv_block(x_input, 384, 1, 1)
conv2_3x3_1x1 = conv_block(conv2_3x3_1x1, 448, 1, 3)
conv2_3x3_1x1 = conv_block(conv2_3x3_1x1, 512, 3, 1)
conv2_3x3_1x1_1 = conv_block(conv2_3x3_1x1, 256, 3, 1)
conv2_3x3_1x1_2 = conv_block(conv2_3x3_1x1, 256, 1, 3)
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv3x3_1, conv3x3_2, conv2_3x3_1x1_1, conv2_3x3_1x1_2], axis=-1)
return merged_vector
def reduction_A(x_input, k=192, l=224, m=256, n=384):
with K.name_scope('Reduction_A'):
"""Architecture of a 35 * 35 to 17 * 17 Reduction_A block."""
maxpool = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
conv3x3 = conv_block(x_input, n, 3, 3, strides=(2, 2), padding='valid')
conv2_3x3 = conv_block(x_input, k, 1, 1)
conv2_3x3 = conv_block(conv2_3x3, l, 3, 3)
conv2_3x3 = conv_block(conv2_3x3, m, 3, 3, strides=(2, 2), padding='valid')
merged_vector = concatenate([maxpool, conv3x3, conv2_3x3], axis=-1)
return merged_vector
def reduction_B(x_input):
"""Architecture of a 17 * 17 to 8 * 8 Reduction_B block."""
with K.name_scope('Reduction_B'):
maxpool = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
conv3x3 = conv_block(x_input, 192, 1, 1)
conv3x3 = conv_block(conv3x3, 192, 3, 3, strides=(2, 2), padding='valid')
conv1x7_7x1_3x3 = conv_block(x_input, 256, 1, 1)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 256, 1, 7)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 320, 7, 1)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 320, 3, 3, strides=(2, 2), padding='valid')
merged_vector = concatenate([maxpool, conv3x3, conv1x7_7x1_3x3], axis=-1)
return merged_vector
def inception_v4_backbone(nb_classes=1000, load_weights=True):
x_input = Input(shape=(299, 299, 3))
# Stem
x = stem(x_input) # 35 x 35 x 384
# 4 x Inception_A
for i in range(4):
x = inception_A(x) # 35 x 35 x 384
# Reduction_A
x = reduction_A(x, k=192, l=224, m=256, n=384) # 17 x 17 x 1024
# 7 x Inception_B
for i in range(7):
x = inception_B(x) # 17 x 17 x1024
# Reduction_B
x = reduction_B(x) # 8 x 8 x 1536
# Average Pooling
x = AveragePooling2D(pool_size=(8, 8))(x) # 1536
# dropout
x = Dropout(0.2)(x)
x = Flatten()(x) # 1536
# 全连接层
x = Dense(units=nb_classes, activation='softmax')(x)
model = Model(inputs=x_input, outputs=x, name='Inception-V4')
return model
if __name__ == '__main__':
inception_v4 = inception_v4_backbone()
plot_model(inception_v4, 'inception_v4.png', show_shapes=True)
4.2 inception_resnet_v1
from keras.layers import Input
from keras.layers.merge import concatenate, add
from keras.layers import Dense, Dropout, Lambda, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from keras import backend as K
from keras.utils import plot_model
RESNET_V1_A_COUNT = 0
RESNET_V1_B_COUNT = 0
RESNET_V1_C_COUNT = 0
def resnet_v1_stem(x_input):
with K.name_scope('Stem'):
x = Conv2D(filters=32, kernel_size=(3, 3), strides=(2, 2), activation='relu', padding='valid')(x_input)
x = Conv2D(32, (3, 3), activation='relu', padding='valid')(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(3, 3), strides=2, padding='valid')(x)
x = Conv2D(80, (1, 1), activation='relu', padding='same')(x)
x = Conv2D(192, (3, 3), activation='relu', padding='valid')(x)
x = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(x)
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_A(x_input, scale_residual=True):
""" 35x35 卷积核"""
global RESNET_V1_A_COUNT
RESNET_V1_A_COUNT += 1
with K.name_scope('resnet_v1_A' + str(RESNET_V1_A_COUNT)):
ar1 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar2 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar2 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar2)
ar3 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar3 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar3)
ar3 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar3)
merged_vector = concatenate([ar1, ar2, ar3], axis=-1)
ar = Conv2D(256, (1, 1), activation='linear', padding='same')(merged_vector)
if scale_residual: # 是否缩小
ar = Lambda(lambda x: 0.1*x)(ar)
x = add([x_input, ar])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_B(x_input, scale_residual=True):
""" 17x17 卷积核"""
global RESNET_V1_B_COUNT
RESNET_V1_B_COUNT += 1
with K.name_scope('resnet_v1_B' + str(RESNET_V1_B_COUNT)):
br1 = Conv2D(128, (1, 1), activation='relu', padding='same')(x_input)
br2 = Conv2D(128, (1, 1), activation='relu', padding='same')(x_input)
br2 = Conv2D(128, (1, 7), activation='relu', padding='same')(br2)
br2 = Conv2D(128, (7, 1), activation='relu', padding='same')(br2)
merged_vector = concatenate([br1, br2], axis=-1)
br = Conv2D(896, (1, 1), activation='linear', padding='same')(merged_vector)
if scale_residual:
br = Lambda(lambda x: 0.1*x)(br)
x = add([x_input, br])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_C(x_input, scale_residual=True):
global RESNET_V1_C_COUNT
RESNET_V1_C_COUNT += 1
with K.name_scope('resnet_v1_C' + str(RESNET_V1_C_COUNT)):
cr1 = Conv2D(192, (1, 1), activation='relu', padding='same')(x_input)
cr2 = Conv2D(192, (1, 1), activation='relu', padding='same')(x_input)
cr2 = Conv2D(192, (1, 3), activation='relu', padding='same')(cr2)
cr2 = Conv2D(192, (3, 1), activation='relu', padding='same')(cr2)
merged_vector = concatenate([cr1, cr2], axis=-1)
cr = Conv2D(1792, (1, 1), activation='relu', padding='same')(merged_vector)
if scale_residual:
cr = Lambda(lambda x: 0.1*x)
x = add([x_input, cr])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def reduction_resnet_A(x_input, k=192, l=224, m=256, n=384):
with K.name_scope('reduction_resnet_A'):
ra1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
ra2 = Conv2D(n, (3, 3), activation='relu', strides=(2, 2), padding='valid')(x_input)
ra3 = Conv2D(k, (1, 1), activation='relu', padding='same')(x_input)
ra3 = Conv2D(l, (3, 3), activation='relu', padding='same')(ra3)
ra3 = Conv2D(m, (3, 3), activation='relu', strides=(2, 2), padding='valid')(ra3)
merged_vector = concatenate([ra1, ra2, ra3], axis=-1)
x = BatchNormalization(axis=-1)(merged_vector)
x = Activation('relu')(x)
return x
def reduction_resnet_B(x_input):
with K.name_scope('reduction_resnet_B'):
rb1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2),padding='valid')(x_input)
rb2 = Conv2D(256, (1, 1), activation='relu', padding='same')(x_input)
rb2 = Conv2D(384, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb2)
rb3 = Conv2D(256, (1, 1),activation='relu', padding='same')(x_input)
rb3 = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb3)
rb4 = Conv2D(256, (1, 1), activation='relu', padding='same')(x_input)
rb4 = Conv2D(256, (3, 3), activation='relu', padding='same')(rb4)
rb4 = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb4)
merged_vector = concatenate([rb1, rb2, rb3, rb4], axis=-1)
x = BatchNormalization(axis=-1)(merged_vector)
x = Activation('relu')(x)
return x
def inception_resnet_v1_backbone(nb_classes=1000, scale=True):
x_input = Input(shape=(299, 299, 3))
# stem
x = resnet_v1_stem(x_input)
# 5 x inception_resnet_v1_A
for i in range(5):
x = inception_resnet_v1_A(x, scale_residual=False)
# reduction_resnet_A
x = reduction_resnet_A(x, k=192, l=192, m=256, n=384)
# 10 x inception_resnet_v1_B
for i in range(10):
x = inception_resnet_v1_B(x, scale_residual=True)
# Reduction B
x = reduction_resnet_B(x)
# 5 x Inception C
for i in range(5):
x = inception_resnet_v1_C(x, scale_residual=True)
# Average Pooling
x = AveragePooling2D(pool_size=(8, 8))(x)
# dropout
x = Dropout(0.2)(x)
x = Flatten()(x)
x = Dense(units=nb_classes, activation='softmax')(x)
return Model(inputs=x_input, outputs=x, name='Inception-Resnet-v1')
if __name__ == '__main__':
inception_resnet_v1_model = inception_resnet_v1_backbone()
plot_model(inception_resnet_v1_model, to_file='inception_resnet_v1.png', show_shapes=True)
4.3 inception_resnet_v2
from keras.layers import Input, add
from keras.layers.merge import concatenate
from keras.layers import Dense, Dropout, Lambda, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from inception_resnet_v1 import reduction_resnet_A
from keras.utils import plot_model
import keras.backend as K
RESNET_V2_A_COUNT = 0
RESNET_V2_B_COUNT = 0
RESNET_V2_C_COUNT = 0
def resnet_v2_stem(x_input):
'''The stem of the pure Inception-v4 and Inception-ResNet-v2 networks. This is input part of those networks.'''
# Input shape is 299 * 299 * 3 (Tensorflow dimension ordering)
with K.name_scope("stem"):
x = Conv2D(32, (3, 3), activation="relu", strides=(2, 2))(x_input) # 149 * 149 * 32
x = Conv2D(32, (3, 3), activation="relu")(x) # 147 * 147 * 32
x = Conv2D(64, (3, 3), activation="relu", padding="same")(x) # 147 * 147 * 64
x1 = MaxPooling2D((3, 3), strides=(2, 2))(x)
x2 = Conv2D(96, (3, 3), activation="relu", strides=(2, 2))(x)
x = concatenate([x1, x2], axis=-1) # 73 * 73 * 160
x1 = Conv2D(64, (1, 1), activation="relu", padding="same")(x)
x1 = Conv2D(96, (3, 3), activation="relu")(x1)
x2 = Conv2D(64, (1, 1), activation="relu", padding="same")(x)
x2 = Conv2D(64, (7, 1), activation="relu", padding="same")(x2)
x2 = Conv2D(64, (1, 7), activation="relu", padding="same")(x2)
x2 = Conv2D(96, (3, 3), activation="relu", padding="valid")(x2)
x = concatenate([x1, x2], axis=-1) # 71 * 71 * 192
x1 = Conv2D(192, (3, 3), activation="relu", strides=(2, 2))(x)
x2 = MaxPooling2D((3, 3), strides=(2, 2))(x)
x = concatenate([x1, x2], axis=-1) # 35 * 35 * 384
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_A(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_A block which is a 35 * 35 grid module.'''
global RESNET_V2_A_COUNT
RESNET_V2_A_COUNT += 1
with K.name_scope('inception_resnet_v2_A' + str(RESNET_V2_A_COUNT)):
ar1 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar2 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar2 = Conv2D(32, (3, 3), activation="relu", padding="same")(ar2)
ar3 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar3 = Conv2D(48, (3, 3), activation="relu", padding="same")(ar3)
ar3 = Conv2D(64, (3, 3), activation="relu", padding="same")(ar3)
merged = concatenate([ar1, ar2, ar3], axis=-1)
ar = Conv2D(384, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: ar = Lambda(lambda a: a * 0.1)(ar)
x = add([x_input, ar])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_B(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_B block which is a 17 * 17 grid module.'''
global RESNET_V2_B_COUNT
RESNET_V2_B_COUNT += 1
with K.name_scope('inception_resnet_v2_B' + str(RESNET_V2_B_COUNT)):
br1 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
br2 = Conv2D(128, (1, 1), activation="relu", padding="same")(x_input)
br2 = Conv2D(160, (1, 7), activation="relu", padding="same")(br2)
br2 = Conv2D(192, (7, 1), activation="relu", padding="same")(br2)
merged = concatenate([br1, br2], axis=-1)
br = Conv2D(1152, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: br = Lambda(lambda b: b * 0.1)(br)
x = add([x_input, br])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_C(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_C block which is a 8 * 8 grid module.'''
global RESNET_V2_C_COUNT
RESNET_V2_C_COUNT += 1
with K.name_scope('inception_resnet_v2_C' + str(RESNET_V2_C_COUNT)):
cr1 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
cr2 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
cr2 = Conv2D(224, (1, 3), activation="relu", padding="same")(cr2)
cr2 = Conv2D(256, (3, 1), activation="relu", padding="same")(cr2)
merged = concatenate([cr1, cr2], axis=-1)
cr = Conv2D(2144, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: cr = Lambda(lambda c: c * 0.1)(cr)
x = add([x_input, cr])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def reduction_resnet_v2_B(x_input):
'''Architecture of a 17 * 17 to 8 * 8 Reduction_ResNet_B block.'''
with K.name_scope('reduction_resnet_v2_B'):
rbr1 = MaxPooling2D((3, 3), strides=(2, 2), padding="valid")(x_input)
rbr2 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr2 = Conv2D(384, (3, 3), activation="relu", strides=(2, 2))(rbr2)
rbr3 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr3 = Conv2D(288, (3, 3), activation="relu", strides=(2, 2))(rbr3)
rbr4 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr4 = Conv2D(288, (3, 3), activation="relu", padding="same")(rbr4)
rbr4 = Conv2D(320, (3, 3), activation="relu", strides=(2, 2))(rbr4)
merged = concatenate([rbr1, rbr2, rbr3, rbr4], axis=-1)
rbr = BatchNormalization(axis=-1)(merged)
rbr = Activation("relu")(rbr)
return rbr
def inception_resnet_v2(nb_classes=1001, scale=True):
'''Creates the Inception_ResNet_v1 network.'''
init = Input((299, 299, 3)) # Channels last, as using Tensorflow backend with Tensorflow image dimension ordering
# Input shape is 299 * 299 * 3
x = resnet_v2_stem(init) # Output: 35 * 35 * 256
# 5 x Inception A
for i in range(5):
x = inception_resnet_v2_A(x, scale_residual=scale)
# Output: 35 * 35 * 256
# Reduction A
x = reduction_resnet_A(x, k=256, l=256, m=384, n=384) # Output: 17 * 17 * 896
# 10 x Inception B
for i in range(10):
x = inception_resnet_v2_B(x, scale_residual=scale)
# Output: 17 * 17 * 896
# Reduction B
x = reduction_resnet_v2_B(x) # Output: 8 * 8 * 1792
# 5 x Inception C
for i in range(5):
x = inception_resnet_v2_C(x, scale_residual=scale)
# Output: 8 * 8 * 1792
# Average Pooling
x = AveragePooling2D((8, 8))(x) # Output: 1792
# Dropout
x = Dropout(0.2)(x) # Keep dropout 0.2 as mentioned in the paper
x = Flatten()(x) # Output: 1792
# Output layer
output = Dense(units=nb_classes, activation="softmax")(x) # Output: 10000
model = Model(init, output, name="Inception-ResNet-v2")
return model
if __name__ == "__main__":
inception_resnet_v2_model = inception_resnet_v2()
plot_model(inception_resnet_v2_model, to_file='inception_resnet_v2.png', show_shapes=True)
InceptionV4的更多相关文章
- 【Network Architecture】Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning（转）
文章来源: https://www.cnblogs.com/shouhuxianjian/p/7786760.html Feature Extractor[Inception v4] 0. 背景 随着 ...
- Tensorflow 训练inceptionV4 并移植
安装brazel (请使用最新版的brazel 和最新版的tensorflow ,版本不匹配会出错!!!) 下载bazel-0.23 https://pan.baidu.com/s/1X ...
- (转) Awesome - Most Cited Deep Learning Papers
转自:https://github.com/terryum/awesome-deep-learning-papers Awesome - Most Cited Deep Learning Papers ...
- awesome-very-deep-learning
awesome-very-deep-learning is a curated list for papers and code about implementing and training ver ...
- 学习笔记TF032:实现Google Inception Net
Google Inception Net,ILSVRC 2014比赛第一名.控制计算量.参数量,分类性能非常好.V1,top-5错误率6.67%,22层,15亿次浮点运算,500万参数(AlexNet ...
- 读论文系列：Deep transfer learning person re-identification
读论文系列:Deep transfer learning person re-identification arxiv 2016 by Mengyue Geng, Yaowei Wang, Tao X ...
- 【深度学习系列】用PaddlePaddle和Tensorflow实现经典CNN网络GoogLeNet
前面讲了LeNet.AlexNet和Vgg,这周来讲讲GoogLeNet.GoogLeNet是由google的Christian Szegedy等人在2014年的论文<Going Deeper ...
- 【深度学习系列】用PaddlePaddle和Tensorflow实现GoogLeNet InceptionV2/V3/V4
上一篇文章我们引出了GoogLeNet InceptionV1的网络结构,这篇文章中我们会详细讲到Inception V2/V3/V4的发展历程以及它们的网络结构和亮点. GoogLeNet Ince ...
- 吐血整理：人工智能PDF中文教材资源包2.73G基本包含全部学习资料-人工智能学习书单
吐血整理:人工智能PDF中文教材资源包2.73G基本包含全部学习资料 人工智能学习书单(关注微信公众号:aibbtcom获取更多资源) 文末附百度网盘下载地址 人工神经网络与盲信号处理 人工神经网络与 ...
随机推荐
- 谈谈我对 Flutter 未来发展 和 “嵌套地狱” 的浅显看法
Flutter 未来发展 提到 Flutter 就不得不提到 Fuchsia 系统,这是一个尚未正式发布的操作的系统,引用 Android 和 Chrome 的高级副总裁 Hiroshi Lockhe ...
- redis基础二----操作List类型
1.lpush的使用方法 2.rpsuh的使用方法 3.删除元素 lrem中2值的是删除2个集合中的“b”元素 4. 通过上面的分析,redis中的list比较类型java的qunue队列
- linux下将多个ts文件合并为一个MP4文件
1. 安装ffmpeg工具 sudo apt install ffmpeg 2. 确保所有ts文件无损坏后,确保当前目录(即存放ts文件的目录)无txt文件及mp4文件,在存放ts文件的目录下建立te ...
- Python实用笔记 （7）高级特性——切片
列表L = ['Michael', 'Sarah', 'Tracy', 'Bob', 'Jack'] 取前3个元素 >>> L[0:3] ['Michael', 'Sarah', ' ...
- AcWing 走廊泼水节 题解
这道题大致题意就是让一棵树任意两点有连边(也就是完全图),但是补完后最小生成树是一开始的那棵树,问最小加的边权之和是多少. 了解题意后,我们可以想到用Kruskal(废话),当每两个集合合并的时候,除 ...
- linux系统，没有安装任何编辑器的情况，如何操作文件
[清空文件] echo " " > filename [添加内容] echo "123" >> filename
- python基础知识-1
1.python是静态的还是动态的?是强类型还弱类型? python是强类型的动态脚本语言: 强类型:不允许不同类型相加 动态:不使用显示类型声明,且确定一个变量的类型是在第一次给它赋值的时候 脚本语 ...
- java语言进阶（七）_Lambda表达式
1 函数式编程思想概述 在数学中,函数就是有输入量.输出量的一套计算方案,也就是"拿什么东西做什么事情".相对而言,面向对象过分强调"必须通过对象的形式来做事情" ...
- Spring IoC 属性赋值阶段
前言 本系列全部基于 Spring 5.2.2.BUILD-SNAPSHOT 版本.因为 Spring 整个体系太过于庞大,所以只会进行关键部分的源码解析. 本篇文章主要介绍 Spring IoC 容 ...
- 这样基于Netty重构RPC框架你不可能知道
原创申明:本文由公众号[猿灯塔]原创,转载请说明出处标注 今天是猿灯塔“365天原创计划”第5天. 今天呢!灯塔君跟大家讲: 基于Netty重构RPC框架 一.CyclicBarrier方法说明 1. ...