Up-sampling in convolutional neural network

21 次查看（过去 30 天）
hoi
hoi 2018-12-26
回答： Reza 2024-6-16
Hi everyone,
for a project at university I am trying to rebuild a NN described in a paper. It was orininally designed in Keras (I don't have any code, only its rough describtion) and I'm struggling with one specific layer they're using. To up-sample their data, they use a layer which takes a single entry of its input and replicates it to a 2x2-region of the output. This results in a matrix with doubled dimensions, without zero-entries (assuming there was none in input) and same entry in each 2x2-block. It is an approximation to the inverse of the maxPooling-Layer of MATLAB. It is similar, but NOT the same as maxUnpooling-Layer, which keeps the position of an maximum-entry and fills up with zeros. For this specific "up-sampling-operation", there is no explicit NN-layer in MATLAB.
Does someone have an idea how I can do this operation?
An idea I had in mind: Just using the given maxUnpooling-Layer and hope there will be no big difference. I tried this and prepared my maxPooling-Layers with "HasUnpoolingOutputs", but it seems that maxUnpooling-Layer has to follow immediately after the maxPooling-Layer. I get unused outputs for my maxPooling-Layers and missing outputs for my maxUnpooling-Layers (seen via analyzeNetwork) as I use convolution-layers in between (see code for example).
layers = [
imageInputLayer([32 32 1])
convolution2dLayer(filterSize, 32, 'Padding', 'same')
batchNormalizationLayer()
reluLayer()
maxPooling2dLayer(2,'Stride',2,'HasUnpoolingOutputs',true) %
convolution2dLayer(filterSize, 64, 'Padding', 'same')
batchNormalizationLayer()
reluLayer()
maxUnpooling2dLayer() %
convolution2dLayer(filterSize, 32, 'Padding', 'same')
batchNormalizationLayer()
reluLayer()
fullyConnectedLayer(32)
regressionLayer
];
So in this case, one has to bring the outputs "indices" and "size" of the maxPooling-layer to the maxUnpooling-layer. But I don't know how this can be achieved :/
I'd be very thankful for any ideas.

请先登录，再回答此问题。

回答（2 个）

Teja Muppirala
Teja Muppirala 2018-12-27
I think you may be able to do it using a transposedConv2dLayer. This works, (though I'm not entirely confident that this is the most appropriate implementation). Define a new layer like this and then use it in your layers instead of the maxUnpooling.
upsample2x2Layer = transposedConv2dLayer(2,1,'Stride',2, 'WeightLearnRateFactor',0,'BiasLearnRateFactor',0);
upsample2x2Layer.Weights = [1 1;1 1];
upsample2x2Layer.Bias = [0];
We set the learn rate factors to zero so that the parameters in this layer are not changed during training.
  2 个评论
hoi
hoi 2018-12-28
编辑：hoi 2018-12-29
Thanks a lot for your answer! Do I have to adjust the third dimension of weights and bias (which characterize my filter) to the actual number of channels (64 in this case)? I'm wondering because I get an 'Input size mismatch'.
Edit: I tried this. I get no more error, so input seems to be right, but the whole stack was reduced to a depth of 1, as the filter is applied to all depths at the same time and summed up to a single number.
Ali Riza Durmaz
Ali Riza Durmaz 2019-4-15
Hello Hoi,
I am dealing with a very similar problem currently. Did the approach of using transposedConv2dLayer with adjusted weight's, biases and their respective learn rate's set to zero solve the problem?
If I understand your last sentence right, you do need a way to apply this to all feature channel's separately so they won't be lost? If I understand the function correctly the second argument 'numFilters' specifies the number of output channels. So additionally to adjusting the weight, and biases you might need to adjust numFilters argument.
Best regards
Adidue

请先登录，再进行评论。

Reza
Reza 2024-6-16
The convenient way is to create a custom layer as explained here, with examples here and here.
For your application, the custom layer is quite simple to implement. It should look something like this:
classdef ndUpsampleLayer < nnet.layer.Layer ...
& nnet.layer.Acceleratable
properties (Learnable)
end
methods
function layer = ndUpsampleLayer(name)
layer.Name = name;
% Set layer description.
layer.Description = "Upsampling as described in the paper";
end
function Y = predict(layer, X) % The main functionality here
% Shape of input X and output Y:
% h-by-w-by-c-by-N numeric array, where h, w, c and N are the
% height, width, number of channels of the images, and number
% of observations, respectively.
sz = size(X);
sy(1:2) = sz(1:2)*2;
Y = zeros(sy,'like',X); % initiate output
Y(1:2:end, 1:2:end, :, :) = X;
Y(1:2:end, 2:2:end, :, :) = X;
Y(2:2:end, 1:2:end, :, :) = X;
Y(2:2:end, 2:2:end, :, :) = X;
end
end
end

类别

在 Help Center 和 File Exchange 中查找有关 Build Deep Neural Networks 的更多信息

标签

产品


版本

R2018b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by