BVLC CaffeNet可视化及类别预测

一、介绍

bvlc_reference_caffenet网络模型是由AlexNet的网络模型改写的，输入图片尺寸大小为227x227x3，输出的为该图片对应1000个分类的概率值。

介绍参考：caffe/models/bvlc_reference_caffenet at master · BVLC/caffe · GitHub  https://github.com/BVLC/caffe/tree/master/models/bvlc_reference_caffenet

二、利用pycaffe可视化网络结构

caffe/python$ python draw_net.py ../models/bvlc_reference_caffenet/deploy.prototxt deploy.png

网络结构：

大图下载地址：链接：https://pan.baidu.com/s/1ggeKlLstZQrOklvnZ03L5A 密码：x7r8

三、matlab可视化

1、网络权值可视化：https://www.cnblogs.com/smbx-ztbz/p/9343874.html

2、特征图可视化

（1）visualize_feature_maps.m

function [] = visualize_feature_maps(w, s)
h = max(size(w, 1), size(w, 2));
g = h + s;
c = size(w, 3);
cv = ceil(sqrt(c));%按长宽相等方式排布，ceil向上取整
W = zeros(g*cv, g*cv);

for u = 1:cv
    for v = 1:cv
        tw = zeros(h, h);
        if (((u-1)*cv + v) <= c)
            tw = w(:, :, (u-1)*cv+v, 1)';%只对第四维度为1进行可视化，即第一个样本进行可视化
            tw = tw - min(min(tw));
            tw = tw / max(max(tw))*255;
        end
        W(g*(u-1) + (1:h), g*(v-1) + (1:h)) = tw;
    end
end
W = uint8(W);
figure, imshow(W);

（2）fm_visual.m

clear;
clc;
close all;
addpath('matlab')
caffe.set_mode_cpu();
sprintf(['Caffe Version = ', caffe.version(), '\n']);
net = caffe.Net('models/bvlc_reference_caffenet/deploy.prototxt',...
'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel', 'test');

sprintf('Load net done. Net layers: ');
net.layer_names

sprintf('Net blobs: ');
net.blob_names

sprintf('Now preparing data...\n');
im = imread('examples/images/cat.jpg');
figure;imshow(im);title('Original Image');
d = load('matlab/+caffe/imagenet/ilsvrc_2012_mean.mat');
mean_data = d.mean_data;%256x256x3
IMAGE_DIM = 256;
CROPPED_DIM = 227;

%Convert an fimage returned by Matlab's imread to im_data in caffe's data
%format: W x H x C with BGR channels
im_data = im(:, :, [3, 2, 1]); %permute channels from RGB to BGR
im_data = permute(im_data, [2, 1, 3]); %flip width and height
im_data = single(im_data); %convert from uint8 to single
im_data = imresize(im_data, [IMAGE_DIM IMAGE_DIM], 'bilinear'); %resize im_data 使得跟mean_data尺寸一致
im_data = im_data - mean_data; % subtract mean_data (already in W x H x C, BGR)
im = imresize(im_data, [CROPPED_DIM CROPPED_DIM], 'bilinear'); %resize im_data
km = cat(4, im, im, im, im, im);%在第四个维度往后叠加，第三维度为1。 227x227x3x5
pm = cat(4, km, km);%在第四个维度往后叠加。 227x227x3x10
input_data = {pm};%输入的数据为输入图片拷贝10份

scores = net.forward(input_data);%cell 1000x10，输入的样本个数为10

scores = scores{1};%指向第一个cell，转换为矩阵
scores = mean(scores, 2); %take average scores over 10 crops，对10个样本求均值

[~, maxlabel] = max(scores);%获取概率均值最大的索引 282

maxlabel %显示所属类别概率最大的下标
figure; plot(scores);

fm_data = net.blob_vec(1);%输入数据
d1 = fm_data.get_data();
sprintf('Data size = ');
size(d1) %227x227x3x10
visualize_feature_maps(d1, 1);

fm_conv1 = net.blob_vec(2);
f1 = fm_conv1.get_data();
sprintf('Feature map conv1 size = ');
%kernel_size: 11, stride: 4, pad: 0 (pad为0表示不对边界进行扩展)
size(f1)%55x55x96x10
visualize_feature_maps(f1, 1);

fm_conv2 = net.blob_vec(5);
f2 = fm_conv2.get_data();
sprintf('Feature map conv2 size = ');
%kernel_size: 5, stride: 1, pad: 2 （步进应该为2？）
size(f2) %27    27   256    10
visualize_feature_maps(f2, 1);

fm_conv3 = net.blob_vec(8);
f3 = fm_conv3.get_data();
sprintf('Feature map conv3 size = ');
%kernel_size: 3, stride: 1, pad: 1 （步进应该为2？）
size(f3)%13    13   384    10
visualize_feature_maps(f3, 1);

fm_conv4 = net.blob_vec(9);
f4 = fm_conv4.get_data();
sprintf('Feature map conv4 size = ');
%kernel_size: 3, stride: 1, pad: 1
size(f4)%13    13   384    10
visualize_feature_maps(f4, 1);

fm_conv5 = net.blob_vec(10);
f5 = fm_conv5.get_data();
sprintf('Feature map conv5 size = ');
%kernel_size: 3, stride: 1, pad: 1
size(f5)%13    13   256    10
visualize_feature_maps(f5, 1);

（3）说明

a、scores为输入图片对应1000个类别的概率值，maxlabel为对应最大概率值的下标，及所输入图像被分为哪一类，得到该图片的最大概率对应的索引为282。

b、类别索引和名称对应表可通过data/ilsvrc12/get_ilsvrc_aux.sh 下载解压，在synset_words.txt文件中，根据行号，来找对应的类别。

四、对输入图片进行类别预测

clear;
clc;
close all;
addpath('matlab')
caffe.set_mode_cpu();
sprintf(['Caffe Version = ', caffe.version(), '\n']);
net = caffe.Net('models/bvlc_reference_caffenet/deploy.prototxt',...
'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel', 'test');

im = imread('examples/images/cat.jpg');
% figure;imshow(im);title('Original Image');
d = load('matlab/+caffe/imagenet/ilsvrc_2012_mean.mat');
mean_data = d.mean_data;%256x256x3
IMAGE_DIM = 256;
CROPPED_DIM = 227;

%Convert an fimage returned by Matlab's imread to im_data in caffe's data
%format: W x H x C with BGR channels
im_data = im(:, :, [3, 2, 1]); %permute channels from RGB to BGR
im_data = permute(im_data, [2, 1, 3]); %flip width and height
im_data = single(im_data); %convert from uint8 to single
im_data = imresize(im_data, [IMAGE_DIM IMAGE_DIM], 'bilinear'); %resize im_data 使得跟mean_data尺寸一致
im_data = im_data - mean_data; % subtract mean_data (already in W x H x C, BGR)
im = imresize(im_data, [CROPPED_DIM CROPPED_DIM], 'bilinear'); %resize im_data
km = cat(4, im, im, im, im, im);%在第四个维度往后叠加，第三维度为1。 227x227x3x5
pm = cat(4, km, km);%在第四个维度往后叠加。 227x227x3x10
input_data = {pm};%输入的数据为输入图片拷贝10份

scores = net.forward(input_data);%cell 1000x10，输入的样本个数为10

scores = scores{1};%指向第一个cell，转换为矩阵
scores = mean(scores, 2); %take average scores over 10 crops，对10个样本求均值

[~, maxlabel] = max(scores);%获取概率均值最大的索引

maxlabel %显示所属类别概率最大的下标
figure; plot(scores);

%打印出对应的label字符串
ffid = fopen('data/ilsvrc12/synset_words.txt','r');
for i = 1:1000
    tline = fgetl(ffid);
    if(i == maxlabel)
%         tline
        break;
    end
end
label_string = tline(11:size(tline, 2));
sprintf('predict value is: %s\n', label_string)
sprintf('probability is: %f\n', scores(maxlabel))

输出：

maxlabel =

   282

ans =

predict value is: tabby, tabby cat

ans =

probability is: 0.288967

可用其他图片进行测试，例如网上下载个熊猫图片进行测试。

参考：

caffe中pad的作用 - CSDN博客  https://blog.csdn.net/xunan003/article/details/79110253

与AlexNet对比：Caffe学习笔记（二）——AlexNet模型 - CSDN博客  https://blog.csdn.net/hong__fang/article/details/52080280

【AlexNet】模型训练与测试导读 - CSDN博客  https://blog.csdn.net/xiequnyi/article/details/52276240?locationNum=5

Caffe下自己的数据训练和测试 - CSDN博客  https://blog.csdn.net/qqlu_did/article/details/47131549

end