GRAD-CAM++のコード作成時のエラー

Question

敬 2024-10-29，11:47

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2162715-grad-cam

评论：敬 2024-11-15，23:50

GRAD-CAMより高解像度の画像の特徴量抽出が可能になるだろうとのことで、下記のようなGRAD-CAM＋＋のコードを作成しましたが、エラーが発生しました。function中の2階微分３階微分のdlgradientのトレースがうまくいっていないようなのです。１階微分のgradientsはトレースできているようです。色々ためしましたが原因と対策が分かりません。どなたかご教授頂けないでしょうか？よろしくお願いいたします。以下コードと発生したエラーです。よろしくお願いいたします。

（コード）

%%%%サポートベクターマシンによる分類 %%%%%%%

%%%%%転移学習ネットワークを選択可能　GRAD-CAM++で可視化

%%%%% 2024年10月27日 2階微分と3階微分の計算でエラー発生中。可能性あり

%データストアの準備

imds2 = imageDatastore("C:\Users\user\MATLAB Drive\Examples\Ultrasonic_analysis\mltest2","IncludeSubfolders",true,"FileExtensions",".png","LabelSource","foldernames");

%動画N増し（５つずつ）

%imds2 = imageDatastore("C:\Users\user\MATLAB Drive\Examples\Ultrasonic_analysis\actual_videono_normalized_cropped","IncludeSubfolders",true,"FileExtensions",".png","LabelSource","foldernames");

%加工した動画

%imds2 = imageDatastore("C:\Users\user\MATLAB Drive\Examples\Ultrasonic_analysis\actual_videono_normalized_cropped3","IncludeSubfolders",true,"FileExtensions",".png","LabelSource","foldernames");

%加工した動画(振動子除去、正規化など）

%imds3 = imageDatastore("C:\Users\user\MATLAB Drive\Examples\Ultrasonic_analysis\actual_videono_normalized_cropped3","IncludeSubfolders",true,"FileExtensions",".png","LabelSource","foldernames");

%ネットワークの準備

%net=googlenet;

%net=efficientnetb0;

%net=inceptionresnetv2;

net=nasnetlarge;

%訓練データとテストデータに分割

[imds2Train,imds2Validation]=splitEachLabel(imds2,0.7,'randomized');

analyzeNetwork(net)

net.Layers

imageAugmenter = imageDataAugmenter( ...

'RandRotation',[-1,1], ...

'RandXTranslation',[-50 50], ...

'RandYTranslation',[50 50], ...

'RandScale',[0.95,1.05], ...

'FillValue',0);

%augdsTrain2= augmentedImageDatastore([224 224 3],imds2Train,'DataAugmentation',imageAugmenter,'ColorPreprocessing','gray2rgb');

%augdsValidation2=augmentedImageDatastore([224 224 3],imds2Validation,'DataAugmentation',imageAugmenter,'ColorPreprocessing','gray2rgb');

%augdsTrain2= augmentedImageDatastore([299 299 3],imds2Train,'DataAugmentation',imageAugmenter,'ColorPreprocessing','gray2rgb');

%augdsValidation2=augmentedImageDatastore([299 299 3],imds2Validation,'DataAugmentation',imageAugmenter,'ColorPreprocessing','gray2rgb');

augdsTrain2= augmentedImageDatastore([331 331 3],imds2Train,'DataAugmentation',imageAugmenter,'ColorPreprocessing','gray2rgb');

augdsValidation2=augmentedImageDatastore([331 331 3],imds2Validation,'DataAugmentation',imageAugmenter,'ColorPreprocessing','gray2rgb');

%%%データの表示と確認

minibatch_Train2 = preview(augdsTrain2);

figure(10)

imshow(imtile(minibatch_Train2.input));

%%%%%%%%%% 機械学習による分類 %%%%%%%%%%

%%%% 特徴の抽出

% 特徴マップを取得する層の名前

%layer = 'activation_295';

featureLayer = 'global_average_pooling2d_2';

%%%特徴量抽出

featuresTrain = activations(net,augdsTrain2,featureLayer,OutputAs="rows");

featuresTest = activations(net,augdsValidation2,featureLayer,OutputAs="rows");

%%% ラベルの取得

TTrain = imds2Train.Labels;

TTest = imds2Validation.Labels;

%%% SVMに当てはめ

rng default

mdl = fitcecoc(featuresTrain,TTrain,'OptimizeHyperparameters','auto',...;

'HyperparameterOptimizationOptions',struct('AcquisitionFunctionName',...;

'expected-improvement-plus'));

%%% テストイメージの分類

Ypreds2 = predict(mdl,featuresTest);

YValidation2=imds2Validation.Labels;

figure(1);

confusionchart(YValidation2,Ypreds2,'RowSummary','row-normalized');

accuracy = mean(Ypreds2 == imds2Validation.Labels);

%%%%% 離型剤の場合 %%%%%

% 画像の読み込みと前処理

imageFiles = ["normalized_frame_vra_12.png", "normalized_frame_vra82_12.png", ...

"normalized_frame_vra90_12.png", "normalized_frame_vra117_12.png", ...

"normalized_frame_vra214_12.png"];

imageDir = "C:\Users\user\MATLAB Drive\Examples\Ultrasonic_analysis\actual_videono_normalized_cropped3\releaseagent\";

inputSize = net.Layers(1).InputSize(1:2);

images = cell(1, numel(imageFiles));

for i = 1:numel(imageFiles)

img = imread(fullfile(imageDir, imageFiles(i)));

img = imresize(img, inputSize);

images{i} = im2double(img);

end

% 画像を4次元配列に変換

dlImages = cat(4, images{:});

dlImages = dlarray(dlImages, 'SSCB');

% ネットワークの層名を一覧表示

layerNames = arrayfun(@(x) x.Name, net.Layers, 'UniformOutput', false);

disp(layerNames);

% Grad-CAMの計算

layerName = 'global_average_pooling2d_2'; % 例として使用する層

classIdx = 1; % 解析したいクラスのインデックス

% Grad-CAM++の計算

gradCAMMaps = cell(1, numel(images));

for i = 1:numel(images)

img = images{i};

dlImg = dlarray(img, 'SSCB');

% ネットワークをLayerGraphに変換

lgraph = layerGraph(net);

% 出力層を削除

lgraph = removeLayers(lgraph, 'ClassificationLayer_predictions');

% ネットワークをdlnetworkに変換

dlnet = dlnetwork(lgraph);

% Grad-CAMの計算

[scoreMap, featureLayer, reductionLayer] = gradCAM(dlnet, dlImg, classIdx);

% Grad-CAM++の重みを計算

[gradients, scores, secondOrderGradients, thirdOrderGradients] = dlfeval(@modelGradients, dlnet, dlImg, classIdx);

%[gradients, scores] = dlfeval(@modelGradients, dlnet, dlImg, classIdx);

% 勾配をリサイズして featureMap のサイズに合わせる

resizedGradients = gradients;

% 重みの計算

%alpha = gradients .^ 2;

%beta = gradients .^ 3;

alpha = secondOrderGradients .^ 2;

beta = thirdOrderGradients .^ 3;

weights = sum(alpha .* beta, [1 2]) ./ (2 * alpha .* beta + 1e-7);

% クラス活性化マップの生成

camMap = sum(scoreMap .* weights, 3);

% マップのリサイズと正規化

camMap = max(camMap, 0);

camMap = camMap / max(camMap(:));

gradCAMMaps{i} = extractdata(camMap);

figure(2)

subplot(1,5,i); %release agent

imshow(img);

hold on

imagesc(gradCAMMaps{i}, 'AlphaData', 0.5);

colorbar

colormap jet

title(sprintf('Grad-CAM++ of 離型剤 image %d', i));

end

function [gradients, scores, secondOrderGradients, thirdOrderGradients] = modelGradients(dlnet, dlImg, classIdx)

% 特定のクラスに対するスコアを計算

scores = predict(dlnet, dlImg);

score = scores(classIdx);

% 損失関数を定義

loss = -sum(score, 'all');

% 勾配を計算

gradients = dlgradient(loss, dlImg);

% 2階微分の計算

gradientsSum = sum(gradients, 'all');

secondOrderGradients = dlgradient(gradientsSum, dlImg);

% 3階微分の計算

secondOrderGradientsSum = sum(secondOrderGradients, 'all');

thirdOrderGradients = dlgradient(secondOrderGradientsSum, dlImg);

end　　

（エラー）

次を使用中のエラー: dlarray/dlgradient (行 105)

微分する値は、トレースされていません。これは、実数のトレース付き dlarray スカラーでなければなりませ

ん。dlfeval によって呼び出される関数内で dlgradient を使用して変数をトレースしてください。

エラー: svm_nasnetlarge_actualvideo_gradcam_plusplus__20241027m>modelGradients (行

175)

secondOrderGradients = dlgradient(gradientsSum, dlImg);

エラー: deep.internal.dlfeval (行 17)

[varargout{1:nargout}] = fun(x{:});

エラー: deep.internal.dlfevalWithNestingCheck (行 19)

[varargout{1:nargout}] = deep.internal.dlfeval(fun,varargin{:});

エラー: dlfeval (行 31)

[varargout{1:nargout}] = deep.internal.dlfevalWithNestingCheck(fun,varargin{:});

エラー: svm_nasnetlarge_actualvideo_gradcam_plusplus__20241027m (行 130)

[gradients, scores, secondOrderGradients, thirdOrderGradients] =

dlfeval(@modelGradients, dlnet, dlImg, classIdx);

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Gayathri 2024-11-5，7:04

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2162715-grad-cam#answer_1540975

编辑：Gayathri 2024-11-8，8:37

在 MATLAB Online 中打开

Hi @敬,

I understand that you are trying to develop a code for GradCAM++ but facing the error

“Error using dlarray/dlgradient (line 105) The values to differentiate are not traced. They must be real, scalar dlarray with tracing. Use dlgradient within a function called by dlfeval to trace variables.”

This is because we have to keep each differentiable in a different function and then call it using the “dlfeval” function. Please refer to the below code for the same.

function [gradients, scores] = modelGradients(dlnet, dlImg, classIdx)
    scores = predict(dlnet, dlImg);
    score = scores(classIdx);
    loss = -sum(score, 'all');
    gradients = dlgradient(loss, dlImg); 
end
function secondOrderGradients= modelGradients2(gradients, dlImg)
    secondOrderGradients = dlgradient(sum(gradients,"all"), dlImg);
end
function thirdOrderGradients= modelGradients3(secondOrderGradients, dlImg)
    thirdOrderGradients = dlgradient(sum(secondOrderGradients,"all"), dlImg);
end

To call these functions use the following lines of codes.

[gradients, scores] = dlfeval(@modelGradients, net, dlImg, label);
secondOrderGradients = dlfeval(@modelGradients2, gradients,dlImg);
thirdOrderGradients = dlfeval(@modelGradients3, secondOrderGradients,dlImg);

The above-mentioned changes would help resolve the issue of gradients not being traced.

For more information on “dlfeval” and “dlgradient” please refer to the following links.

Hope you find this information helpful.

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

敬 2024-11-15，23:50

Thank you for your kind and insightful answer. I'll definitely try it out.

请先登录，再进行评论。

GRAD-CAM++のコード作成時のエラー

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

GRAD-CAM++のコード作成時のエラー

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

1 个评论 显示 -1更早的评论隐藏 -1更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

1 个评论
显示 -1更早的评论隐藏 -1更早的评论