1D-Convolution Layer not supported by calibrate function

Question

Silvia 2024-10-14

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2159490-1d-convolution-layer-not-supported-by-calibrate-function

编辑： Silvia 2024-11-6，8:20

Good morning,

I am trying to follow this example: https://it.mathworks.com/help/coder/ug/generate-code-for-quantized-lstm-network-and-deploy-on-cortex-m-target.html on how to generate an Int8 Code for an implementation in a STM32.

My network is composed by the following layers:

6×1 Layer array with layers:

1 'input' Sequence Input Sequence input with 1 dimensions

2 'conv1' 1-D Convolution 10 8×1 convolutions with stride 1 and padding 'same'

3 'batchnorm1' Batch Normalization Batch normalization with 10 channels

4 'relu1' ReLU ReLU

5 'gru1' Projected Layer Projected GRU with 32 hidden units

6 'output' Projected Layer Projected fully connected layer with output size 1

When I try to calibrate the network as described in the example, I have the following error showing that the 1D-convolutional layer is not supported in the CPU environment: "Code generation for conv1 is not supported for target library 'mkldnn'. See documentation for a list of supported layers with each target library."

Can I solve this problem without having to change the 1D-convolutional layer?

Thank you in advance,

Silvia

6 个评论
显示 4更早的评论隐藏 4更早的评论

Silvia 2024-10-14

在 MATLAB Online 中打开

Hello @Jayanti, thank you for your help.

Yes, I am trying to adapt the example to my situation, in which I have a custom network.

I cannot provide you the data set, but I can still show you the pipeline I followed. The initial trained network was then compressed with the projection technique.

%% Load Training - Validation Set
load trainingset.mat trainingset;
datasetSize = size(trainingset,2);
trainsetSize = 0.9 * datasetSize; % leaving 10% for Validation
folder      = "---";
% Folder del dataset
dataDir = string(folder + "\dataset");
% For Training
if ~exist(fullfile(dataDir,"train"),'dir')
    mkdir(fullfile(dataDir,"train"))
    cnt = 0;
    % Training set generation
    for idx = 1:trainsetSize
        cnt = cnt + 1;
        cleanSignal = trainingset(idx).cleansignal;   % it has to be in the form: number of samples x 1 !
        noisySignal = trainingset(idx).noisysignal;
    
        save(fullfile(dataDir,"train", ...
                "data_" + num2str(cnt) + ".mat"), ...
                "cleanSignal","noisySignal");
    end
    % Prepare Datastores to Consume Data
    ds_Train = signalDatastore(fullfile(dataDir,"train"), ...
    SignalVariableNames=["noisySignal","cleanSignal"], ...
    ReadOutputOrientation="row");
else
    % Directly prepare Datastores to Consume Data (without training
    % generation)
    ds_Train = signalDatastore(fullfile(dataDir,"train"), ...
    SignalVariableNames=["noisySignal","cleanSignal"], ...
    ReadOutputOrientation="row");
end
% For Validation
cnt = 0;
if ~exist(fullfile(dataDir,"validate"),'dir')
    mkdir(fullfile(dataDir,"validate"))
    % Validation set generation
    for idx = trainsetSize+1 : datasetSize
        cnt = cnt + 1;
        cleanSignal = trainingset(idx).cleansignal;
        noisySignal = trainingset(idx).noisysignal;
    
        save(fullfile(dataDir,"validate", ...
                "data_" + num2str(cnt) + ".mat"), ...
                "cleanSignal","noisySignal");
    end
    % Prepare Datastores to Consume Data
    ds_Validate = signalDatastore(fullfile(dataDir,"validate"), ...
    SignalVariableNames=["noisySignal","cleanSignal"], ...
    ReadOutputOrientation="row");
else
    % Directly prepare Datastores to Consume Data (without valisation set
    % generation)
    ds_Validate = signalDatastore(fullfile(dataDir,"validate"), ...
    SignalVariableNames=["noisySignal","cleanSignal"], ...
    ReadOutputOrientation="row");
end
data = preview(ds_Train);
%% Define the Neural Network Architecture
numHiddenUnits = 32;
% Hybrid Approach ()
layers = [
    sequenceInputLayer(1, 'Name', 'input')
    convolution1dLayer(8, 10, 'Padding', 'same', 'Name', 'conv1')
    batchNormalizationLayer('Name', 'batchnorm1')
    reluLayer('Name', 'relu1')
    gruLayer(numHiddenUnits, 'OutputMode', 'sequence', 'Name', 'gru1')
    fullyConnectedLayer(1, 'Name', 'output')
];
  
%% Specify Training Options
miniBatchSize = 64;
options = trainingOptions('adam', ...            % Use the Adam optimizer
    'MaxEpochs', 30, ...                         % Max number of epochs
    'InitialLearnRate', 1e-4, ...                % Initial learning rate
    'MiniBatchSize', miniBatchSize, ...          % Size of the batches
    'Shuffle', 'every-epoch', ...                % Shuffle the data every epoch
    'ValidationData', ds_Validate, ...           % Validation data
    'Plots', 'training-progress', ...            % Plot training progress
    'Verbose', false, ...                        % Suppress verbose output 
    'ValidationFrequency', floor(trainsetSize/miniBatchSize), ...
    'Metrics', 'rmse');                          % Evaluate using 'rmse'
%% Train the Network
% Ask the user to select an option
userChoice = input('Enter "Train" to train the model or "Download" to download pre-trained model: ', 's');
% Define network name or load network
if userChoice == "Train"
    % Ask the user for the new network name
    netName = input('Enter the name to save the trained network as (e.g., "LSTM01"): ', 's');
    netFile = [netName, '.mat'];
elseif userChoice == "Download"
    % Open file selection dialog to choose the network file
    [fileName, filePath] = uigetfile('*.mat', 'Select the pre-trained network file');
    if isequal(fileName, 0)
        disp('No file selected. Exiting...');
        return
    else
        netFile = fullfile(filePath, fileName);
    end
else
    error('Invalid choice. Please enter either "Train" or "Download".');
end
% If train is selected, train the network
if userChoice == "Train"
    % Train the model and save it with the specified name
    disp("Training the network...");
    trainedNet = trainnet(ds_Train, layers, 'mse', options);
    save(netFile, 'trainedNet');
elseif userChoice == "Download"
    % Load the specified pre-trained network
    if isfile(netFile)
        disp("Loading the pre-trained network...");
        load(netFile, 'trainedNet');
    else
        error(['The specified network file "' netFile '" does not exist.']);
    end
else
    error('Invalid choice. Please enter either "Train" or "Download".');
end
netOriginal = trainedNet;
%% Compressed network via PCA (reduction of 55%)
% Create a mini-batch queue object 
numSignals = length(trainingset);
XTrain = cell(numSignals, 1);
for i = 1:numSignals
    noisySignal = trainingset(i).noisysignal';
    XTrain{i} = noisySignal;
end
% Input must be a datastore
adsXTrain = arrayDatastore(XTrain, 'OutputType', 'same');
miniBatchSize = 16;
mbqTrain = minibatchqueue(adsXTrain, ...
    MiniBatchSize=miniBatchSize, ...
    MiniBatchFcn=@preprocessMiniBatchPredictors, ...
    MiniBatchFormat="CTB");
% Create the neuronPCA object
npca = neuronPCA(netOriginal,mbqTrain,VerbosityLevel="steps");
npca
LearnablesReductionValue = 0.55;
% Thus we will set the LearnablesReduction to 55%
netProjected = compressNetworkUsingProjection(netOriginal,npca,...
    LearnablesReduction=LearnablesReductionValue,...
    VerbosityLevel="off");
%% Preparing data for quantization
load calibrationset.mat calibrationset;
calibrationsetSize = size(calibrationset,2);
if ~exist(fullfile(dataDir,"calibration"),'dir')
    mkdir(fullfile(dataDir,"calibration"))
    cnt = 0;
    % Calibration set generation
    for idx = 1:calibrationsetSize
        cnt = cnt + 1;
        cleanSignal = calibrationset(idx).cleansignal;
        noisySignal = calibrationset(idx).noisysignal;
    
        save(fullfile(dataDir,"calibration", ...
                "data_" + num2str(cnt) + ".mat"), ...
                "cleanSignal","noisySignal");
    end
    % Prepare Datastores to Consume Data
    ds_Cal = signalDatastore(fullfile(dataDir,"calibration"), ...
    SignalVariableNames=["noisySignal","cleanSignal"], ...
    ReadOutputOrientation="row");
else
    % Directly prepare Datastores to Consume Data (without training
    % generation)
    ds_Cal = signalDatastore(fullfile(dataDir,"calibration"), ...
    SignalVariableNames=["noisySignal","cleanSignal"], ...
    ReadOutputOrientation="row");
end
% Adapt Calibration Set to the mini-batch queue specification
numSignals = length(calibrationset);
XCal = cell(numSignals, 1);
for i = 1:numSignals
    noisySignal = calibrationset(i).noisysignal';
    XCal{i} = noisySignal;
end
%disp(size(XCal));       % Should be [1000, 1]
%disp(size(XCal{1}));    % Should be [1, Nsamples]
% Input must be a datastore
adsXCal = arrayDatastore(XCal, 'OutputType', 'same');
mbqCal = minibatchqueue(adsXCal, ...
    MiniBatchSize=miniBatchSize, ...
    MiniBatchFcn=@preprocessMiniBatchPredictors, ...
    MiniBatchFormat="CTB");
% Quantization
quantObj = dlquantizer(netProjected, 'ExecutionEnvironment', 'CPU');
quantObj.calibrate(adsXCal)

Silvia 2024-11-5，13:27

在 MATLAB Online 中打开

Thank you @Hariprasad Ravishankar and sorry for the late reply.

I am trying to follow the first possible approach in the example (Generate PIL Executable That Accepts a Single Observation of Variable Sequence Length).

% Create a Code Configuration Object
cfg = coder.config('lib','ecoder',true);
% Configure Object for PIL (processor-in-the-loop) Execution
cfg.VerificationMode = 'PIL';
% Specify 'None' as the TargetLibrary when creating a DeepLearningConfig to
% specify no third-parties deep learning libraries (Generate plain C):
cfg.DeepLearningConfig = coder.DeepLearningConfig('TargetLibrary', 'none');
% Specify Target Hardware
cfg.Hardware = coder.hardware('STM32 Nucleo F401RE'); 
% Set PIL Communication Interfance (a serial PIL communication interface)
cfg.Hardware.PILInterface = 'Serial';
% Determine the COM port for serial communication
cfg.Hardware.PILCOMPort = 'COM2';
% Limit stack size because the default stack size is much larger than the
% available memory on the hardware. Set to a smaller value (try with 512)
cfg.StackUsageMax = 512;
% View the log
cfg.Verbose = 1;
% Specify the Code Replacement Library for Cortex-M
cfg.CodeReplacementLibrary = 'ARM Cortex-M (CMSIS)';
%% Generate PIL Executable that accepts a single observation of variable sequence length
% Type function loads the network (in .mat file) into a persistent variable
% The function reuses this persistent object on subsequent prediction calls
type('FinalFineTuned_predict.m')
% Specify input type and size of the input argument tp the codegen command
% by using the coder.typeof function
noisyInputType = coder.newtype('double', [Inf 1], [1 0]);
% Run the codegen command to generate code and PIL executable
codegen -config cfg FinalFineTuned_predict -args {noisyInputType} -report

And my FinalFineTuned_predict.m function is the following:

function out = FinalFineTuned_predict(in) %#codegen
% A persistent object mynet is used to load the series network object.
% At the first call to this function, the persistent object is constructed and
% setup. When the function is called subsequent times, the same object is reused 
% to call predict on inputs, thus avoiding reconstructing and reloading the
% network object.
% Copyright 2019-2021 The MathWorks, Inc. 
persistent mynet;
if isempty(mynet)
    mynet = coder.loadDeepLearningNetwork('FinalFineTuned.mat');
end
% pass in input   
out = predict(mynet, in);

When I run the code, I have the following error:

"### Compiling function(s) FinalFineTuned_predict ...

### Generating compilation report ...

Input data argument to predict must be dlarray type.

Error in ==> FinalFineTuned_predict Line: 18 Column: 7

Code generation failed: View Error Report"

To decide the input type, I used the coderTypeEditor giving as input a variable NoisySignal (7716x1 double) and specifying that I don't want to fix the length of the signal (thus [Inf 1] instead of [7716 1]). If I normally use the predict function in a MATLAB script with this NoisySignal (CleanSignal = predict(netFineTuned, NoisySignal)) no errors occur. Do you know what I am missing?

Thank you in advance,

Silvia

Hariprasad Ravishankar 2024-11-5，14:39

在 MATLAB Online 中打开

Hi @Silvia,

For code generation we expect the input to predict to be a dlarray. Please try modifying your function as follows:

function out = FinalFineTuned_predict(in) %#codegen
% A persistent object mynet is used to load the series network object.
% At the first call to this function, the persistent object is constructed and
% setup. When the function is called subsequent times, the same object is reused 
% to call predict on inputs, thus avoiding reconstructing and reloading the
% network object.
% Copyright 2019-2021 The MathWorks, Inc. 
persistent mynet;
if isempty(mynet)
    mynet = coder.loadDeepLearningNetwork('FinalFineTuned.mat');
end
% pass in input 
% We first cast the 'double' input to 'single' as code-generation only supports 'single' precision compute for dlnetwork.
% We specify the format of input as 'TC' to indicate that the first
% dimension is 'Time' and second dimension is 'channel'.
outDlarray = predict(mynet, dlarray(single(in), 'TC'); 
% We extract data from the dlarray to get back the result in 'single'
% datatype.
out = extractdata(outDlarray);
end

Silvia 2024-11-5，15:30

编辑：Silvia 2024-11-6，8:20

Thank you very much, this solved my error!

Now I am encountering another error: "ProjectedLayer objects not supported for code generation. Unpack the network first using unpackProjectedLayers."

Since I would need to compare the inference times of my models (Original, Fine-Tuned and Projected), if I now unpack the Projected layer, I cannot compare the three models?

请先登录，再进行评论。

请先登录，再回答此问题。

1D-Convolution Layer not supported by calibrate function

6 个评论
显示 4更早的评论隐藏 4更早的评论

回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

1D-Convolution Layer not supported by calibrate function

6 个评论 显示 4更早的评论隐藏 4更早的评论

回答（0 个）

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

6 个评论
显示 4更早的评论隐藏 4更早的评论