error compiling ResNet50 Deep Learning HDL Toolbox Support Package

1 次查看（过去 30 天）
Rubén
Rubén 2023-4-14
评论： Rubén 2023-5-3
Hello,
I have a pretrained ResNet50 imported to a DAGNetwork object. The NN is working properly in matlab. However, I'm trying to implement it in a FPGA and I get the followong error while compiling. NN was imported using importTensorFlowNetwork and I'm trying to generate HDL with Deep Learning HDL Toolbox Support Package For Intel FPGA And SoC Devices (I was able to implement a sequential NN using a SeriesNetwork object and this tool):
Error in dnnfpga.apis.Workflow/compile
Error in generate_HDL_from_NN (line 18)
hW.compile;
Caused by:
Layer 'conv2_block1_3_conv': Input size mismatch. Size of input to this layer is different from the expected input size.
Inputs to this layer:
from layer 'conv2_block1_0_conv' (size 8(S) × 8(S) × 256(C) × 1(B))
Layer 'conv2_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv2_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv2_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block4_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block4_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block5_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block6_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Do you have any suggestion to solve this issue?
Thank you in advance
---------------------------
Software version:
Deep Learning HDL Toolbox Support Package For Intel FPGA And SoC Devices 21.2.4
Matlab R2021b Update 6 (9.11.0.2207237)
Ubuntu 20.04
---------------------------

请先登录，再回答此问题。

回答（2 个）

Wang Chen
Wang Chen 2023-4-16
编辑：Wang Chen 2023-4-16
Hi Ruben,
For Resnet50 network, have you tried to download the Deep Learning Toolbox Model for ResNet-50 Network support pacakge?The Resnet50 network in this support package works together with Deep Learning HDL Toolbox:
https://www.mathworks.com/matlabcentral/fileexchange/64626-deep-learning-toolbox-model-for-resnet-50-network
There is also a ResNet-18 network support pacakge:
https://www.mathworks.com/matlabcentral/fileexchange/68261-deep-learning-toolbox-model-for-resnet-18-network
For the network you imported using importTensorFlowNetwork, could you contact MathWorks customer support and shared the network with customer support? It looks like the network is not correctly constructed in MATLAB. The error message on the "Unconnected input. Each layer input must be connected to the output of another layer" are generic MATLAB network connection errors. They are not specific to FPGA targeting.
Thanks,
Wang
  6 个评论
显示 4更早的评论
Rubén
Rubén 2023-4-17
analyzeNetwork does not give neither warnings nor errors. In addtion nn imported seems working properly, it classify correctly.
My NN uses ResNet50, that is standard and is supported by HDL toolbox, and keras flatten, reLu and softmax layers, that all of them are also supported.
I don't see the point to train Resnet50 out of Matlab and inclue additional layers in Matlab because my model need the additional layers and the training will be different.
Wang Chen
Wang Chen 2023-4-17
Hi Ruben,
It will be great if you can share your imported network to us, so we can take a look at it and see what is the cause of the issue. Could you contact MathWorks customer support and shared the network with customer support?
Alternatively, as Sivylla suggested, you could add the additional desired layers to the MATLAB built-in ResNet50 or ResNet18, using MATLAB tools like Deep Network Designer:
https://www.mathworks.com/help/deeplearning/gs/get-started-with-deep-network-designer.html
Thanks,
Wang

请先登录，再进行评论。

Rubén
Rubén 2023-4-17
The size of the NN is over the 5MB limit. Please, find the "NN.mat" in the following download link: NN.mat
  5 个评论
显示 3更早的评论
Wang Chen
Wang Chen 2023-4-28
Hi Ruben, the root cause of this bug is the pattern of zero padding layer (nnet.keras.layer.ZeroPadding2dLayer) followed by a group convolution layer, please see following bug report: https://www.mathworks.com/support/bugreports/2661508
This bug is fixed in R2022b.
If you cannot upgrade to R2022b, the work-around is to merge the zero padding into the group convolution layer. These two layers can be merged because the group convolution layer also has the padding options.
Thanks,
Wang
Rubén
Rubén 2023-5-3
Hello @Wang Chen,
I have imported the SaveModel as a layerGraph ("layers" in this example) object and I tried to merge both layers in this way:
%% Merge 0 padding layer into groupedConvolution2dLayer layer
% Copy conv layer and change Padding size
conv = layers.Layers(3);
conv.PaddingSize = [layers.Layers(2).Top layers.Layers(2).Bottom layers.Layers(2).Left layers.Layers(2).Right];
% Replace conv layer and remove padding layer
layers = replaceLayer(layers, layers.Layers(3).Name, conv);
layers = removeLayers(layers, layers.Layers(2).Name);
% Connect input layer with conv layer
layers = connectLayers(layers, layers.Layers(1).Name, layers.Layers(2).Name)
NN = assembleNetwork(layers);
after doing this, I have built the DAGNetwork object using assembleNetwork and after analising it with analyzeNetwork there is no error/warning.
However, I get the following error whe I try to compile with hW.compile as before:
Error using dnnfpga.compiler.codegenfpga
Invalid network.
Error in dnnfpga.apis.Workflow/compileNetwork
Error in dnnfpga.apis.Workflow/compile
Error in generate_HDL_from_NN (line 18)
hW.compile;
Error in importTensorFlowNN (line 36)
hW = generate_HDL_from_NN(nn, ToolName, ToolPath, Vendor, Platform, ip, Interface, User, Pass);
Caused by:
Layer 'conv2_block1_3_conv': Input size mismatch. Size of input to this layer is different from the expected input size.
Inputs to this layer:
from layer 'conv2_block1_0_conv' (size 8(S) × 8(S) × 256(C) × 1(B))
Layer 'conv2_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv2_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv2_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv3_block4_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block4_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block5_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv4_block6_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block1_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block2_add': Unconnected input. Each layer input must be connected to the output of another layer.
Layer 'conv5_block3_add': Unconnected input. Each layer input must be connected to the output of another layer.
Find the nn int this link NN_without_padding.mat
Are both layers merged properly?

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 System Integration of Deep Learning Processor IP Core 的更多信息

标签

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by