Previously accessible file is now inaccessible.

8 次查看（过去 30 天）
Cloud Wind
Cloud Wind 2022-3-22
回答： Richard 2022-3-24
Hi everyone, I was using deep learning toolbox to train a CNN. After I edit my code (which can successfully run), it shows the following wrong message "Previously accessible file "inmem:///deep_learning/tpc723775a_5df4_4b61_96f1_552a4a17e7e2.m" is now inaccessible."
Previously accessible file "inmem:///deep_learning/tp7a8bcbea_11cf_4603_8ba8_f6278a23f4fa.m" is now inaccessible.
Error in deep.internal.recording.convert.tapeToFunction>@(varargin)fcnWithConstantsInput(varargin{:},constants) (line 37)
fcn = @(varargin)fcnWithConstantsInput(varargin{:},constants);
Error in deep.internal.AcceleratedOp/backward (line 69)
[varargout{1:op.NumGradients}] = backwardFun(varargin{:});
Error in deep.internal.recording.RecordingArray/backwardPass (line 89)
grad = backwardTape(tm,{y},{initialAdjoint},x,retainData,false,0);
Error in dlarray/dlgradient (line 132)
[grad,isTracedGrad] = backwardPass(y,xc,pvpairs{:});
Error in test_modelGradients320 (line 14)
[gradientsSubnet,gradientsParams] = dlgradient(loss,dlnet.Learnables,fcParams);
Error in deep.internal.dlfeval (line 17)
[varargout{1:nargout}] = fun(x{:});
Error in dlfeval (line 40)
[varargout{1:nargout}] = deep.internal.dlfeval(fun,varargin{:});
Error in test_siamese320 (line 126)
[gradientsSubnet,gradientsParams,loss] = dlfeval(@test_modelGradients320,dlnet,fcParams,dlX1,dlX2,pairLabels,name);
Actually, I didn't find so named file existing anywhere in my computer. I have searched and tried all potential solutions but still cannot solve this problem. The most terrible thing is that it still doesn't work even if I rewrite the code in a new script.
I was using win10+matlab 2021b. Hope for any helpful solutions, thank you!
  8 个评论
显示 6更早的评论
Cloud Wind
Cloud Wind 2022-3-23
编辑：Cloud Wind 2022-3-24
@Richard Thanks Richard. It works by specifying ("Acceleration", "none") . But the final performance is a bit worse.
I post my script and functions. Hope they can help you find the problem!
The main script file and the function files called in the script:
clc; clear;
downloadFolder = 'F:\exps\siamese_scd\data';
dataFolderTrain = fullfile(downloadFolder,'train');
dataFolderTest = fullfile(downloadFolder,'test');
%***************************************************
net = alexnet;
layers=net.Layers(1:22);
layers=[layers(1:18,:);layers(20:21,:)];
lgraph = layerGraph(layers);
dlnet = dlnetwork(lgraph);
fcWeights = dlarray(0.01*randn(1,4352));
fcBias = dlarray(0.01*randn(1,1));
fcParams = struct(...
"FcWeights",fcWeights,...
"FcBias",fcBias);
clear net layers
%***************************************************
imdsTrain = imageDatastore(dataFolderTrain, ...
'IncludeSubfolders',true, ...
'LabelSource','none');
files = imdsTrain.Files;
parts = split(files,filesep);
labels = join(parts(:,(end-2):(end-1)),'_');
imdsTrain.Labels = categorical(labels);
imdsTest = imageDatastore(dataFolderTest, ...
'IncludeSubfolders',true, ...
'LabelSource','none');
files = imdsTest.Files;
parts = split(files,filesep);
labels = join(parts(:,(end-2):(end-1)),'_');
imdsTest.Labels = categorical(labels);
%***************************************************
numIterations =1000;
train_miniBatchSize =8;
test_minBatchSize = 4;
learningRate = 2e-5;
trailingAvgSubnet = [];
trailingAvgSqSubnet = [];
trailingAvgParams = [];
trailingAvgSqParams = [];
gradDecay = 0.9;
gradDecaySq = 0.99;
executionEnvironment = "gpu";
plots = "training-progress";
if plots == "training-progress"
figure
subplot(2,1,1)
trainingPlotAxes = gca;
% trainingPlotAxes.YLim = [0 1];
lineLossTrain = animatedline(trainingPlotAxes);
xlabel(trainingPlotAxes,"Iteration")
ylabel(trainingPlotAxes,"Loss")
title(trainingPlotAxes,"Loss During Training")
subplot(2,1,2)
testingPlotAxes = gca;
% testingPlotAxes.YLim = [0 1];
lineLosstest = animatedline(testingPlotAxes);
xlabel(testingPlotAxes,"Iteration")
ylabel(testingPlotAxes,"Loss")
title(testingPlotAxes,"Loss During Testing")
end
%***************************************************
for iteration = 1:numIterations
[X1,X2,pairLabels] = getAlexnetBatch(imdsTrain,train_miniBatchSize);
[tX1,tX2,tpairLabels] = getAlexnetTest(imdsTest,test_minBatchSize);
dlX1 = dlarray(single(X1),'SSCB');
dlX2 = dlarray(single(X2),'SSCB');
tdlX1 = dlarray(single(tX1),'SSCB');
tdlX2 = dlarray(single(tX2),'SSCB');
if executionEnvironment == "gpu"
dlX1 = gpuArray(dlX1);
dlX2 = gpuArray(dlX2);
tdlX1 = gpuArray(tdlX1);
tdlX2 = gpuArray(tdlX2);
end
[gradientsSubnet,gradientsParams,loss] = dlfeval(@modelGradients,dlnet,fcParams,dlX1,dlX2,pairLabels);
lossValue = double(gather(extractdata(loss)));
[~,~,tloss] = dlfeval(@modelGradients,dlnet,fcParams,tdlX1,tdlX2,tpairLabels);
tlossValue = double(gather(extractdata(tloss)));
clear dlX1 dlX2 tdlX1 tdlX2
%***************************************************
[dlnet,trailingAvgSubnet,trailingAvgSqSubnet] = ...
adamupdate(dlnet,gradientsSubnet, ...
trailingAvgSubnet,trailingAvgSqSubnet,iteration,learningRate,gradDecay,gradDecaySq);
[fcParams,trailingAvgParams,trailingAvgSqParams] = ...
adamupdate(fcParams,gradientsParams, ...
trailingAvgParams,trailingAvgSqParams,iteration,learningRate,gradDecay,gradDecaySq);
if plots == "training-progress"
addpoints(lineLossTrain,iteration,lossValue);
addpoints(lineLosstest,iteration,tlossValue);
end
drawnow;
temp1=sprintf('iteration: %d ----- %d',[iteration,numIterations]);
temp2=sprintf('loss: Training：%0.4f ----- Testing：%0.4f',[lossValue,tlossValue]);
disp(temp1);
disp(temp2);
end
save siamese_AlexNet dlnet fcParams iteration lossValue tlossValue
%******************************************************************************************
% the called functions
%******************************************************************************************
function [gradientsSubnet,gradientsParams,loss] = modelGradients(dlnet,fcParams,dlX1,dlX2,pairLabels)
% Pass the image pair through the network
Y = forwardSiamese(dlnet,fcParams,dlX1,dlX2);
% Calculate binary cross-entropy loss
loss = binarycrossentropy(Y,pairLabels);
% Calculate gradients of the loss with respect to the network learnable
% parameters
[gradientsSubnet,gradientsParams] = dlgradient(loss,dlnet.Learnables,fcParams);
end
function loss = binarycrossentropy(Y,pairLabels)
% binarycrossentropy accepts the network's prediction Y, the true
% label, and pairLabels, and returns the binary cross-entropy loss value.
% Get precision of prediction to prevent errors due to floating
% point precision
precision = underlyingType(Y);
% Convert values less than floating point precision to eps.
Y(Y < eps(precision)) = eps(precision);
%convert values between 1-eps and 1 to 1-eps.
Y(Y > 1 - eps(precision)) = 1 - eps(precision);
% Calculate binary cross-entropy loss for each pair
loss = -pairLabels.*log(Y) - (1 - pairLabels).*log(1 - Y);
% Sum over all pairs in minibatch and normalize.
loss = sum(loss)/numel(pairLabels);
end
%***************************************************************************************
function Y_s = forwardSiamese(dlnet,fcParams,dlX1,dlX2)
% forwardSiamese accepts the network and pair of training images, and returns a
% prediction of the probability of the pair being similar (closer to 1) or
% dissimilar (closer to 0). Use forwardSiamese during training.
% Pass the first image through the twin subnetwork
F1 = forward(dlnet,dlX1);
F1 = sigmoid(F1);
% Pass the second image through the twin subnetwork
F2 = forward(dlnet,dlX2);
F2 = sigmoid(F2);
% Subtract the feature vectors
Y = abs(F1 - F2);
% compute distance map
F1_conv = forward(dlnet,dlX1,'Outputs','relu5');
F1_conv = sigmoid(F1_conv);
F1_conv = reshape(F1_conv,[size(F1_conv,1)*size(F1_conv,2),size(F1_conv,3),size(F1_conv,4)]);
F2_conv = forward(dlnet,dlX2,'Outputs','relu5');
F2_conv = sigmoid(F2_conv);
F2_conv = reshape(F2_conv,[size(F2_conv,1)*size(F2_conv,2),size(F2_conv,3),size(F2_conv,4)]);
D_X1_X2=abs(F1_conv-F2_conv);
D_X1_X2=max(D_X1_X2,[],1);
D_X1_X2=squeeze(D_X1_X2);
% fuse local and global similarity
Y_s=[D_X1_X2;Y];
Y_s = dlarray(Y_s,'CB');
Y_s = sigmoid(Y_s);
% Pass the result through a fullyconnect operation
Y_s = fullyconnect(Y_s,fcParams.FcWeights,fcParams.FcBias);
% Convert to probability between 0 and 1.
Y_s = sigmoid(Y_s);
end
%***************************************************************************************
function [X1,X2,pairLabels] = getAlexnetTest(imds,miniBatchSize)
pairLabels = zeros(1,miniBatchSize);
X1 = zeros([227 227 3 miniBatchSize]);
X2 = zeros([227 227 3 miniBatchSize]);
imdsaug = augmentedImageDatastore([227 227],imds);
batch=readall(imdsaug);
for i = 1:miniBatchSize
choice = rand(1);
if choice < 0.5
[pairIdx1,pairIdx2,pairLabels(i)] = getSimilarPair(batch.response);
else
[pairIdx1,pairIdx2,pairLabels(i)] = getDissimilarPair(batch.response);
end
X1(:,:,:,i) =batch.input{pairIdx1};
X2(:,:,:,i) =batch.input{pairIdx2};
end
end
function [pairIdx1,pairIdx2,pairLabel] = getSimilarPair(classLabel)
% getSimilarSiamesePair returns a random pair of indices for images
% that are in the same class and the similar pair label = 1.
% Find all unique classes.
classes = unique(classLabel);
% Choose a class randomly which will be used to get a similar pair.
classChoice = randi(numel(classes));
% Find the indices of all the observations from the chosen class.
idxs = find(classLabel==classes(classChoice));
% Randomly choose two different images from the chosen class.
pairIdxChoice = randperm(numel(idxs),2);
pairIdx1 = idxs(pairIdxChoice(1));
pairIdx2 = idxs(pairIdxChoice(2));
pairLabel = 1;
end
function [pairIdx1,pairIdx2,label] = getDissimilarPair(classLabel)
% Find all unique classes.
classes = unique(classLabel);
% Choose two different classes randomly which will be used to get a dissimilar pair.
classesChoice = randperm(numel(classes),2);
% Find the indices of all the observations from the first and second classes.
idxs1 = find(classLabel==classes(classesChoice(1)));
idxs2 = find(classLabel==classes(classesChoice(2)));
% Randomly choose one image from each class.
pairIdx1Choice = randi(numel(idxs1));
pairIdx2Choice = randi(numel(idxs2));
pairIdx1 = idxs1(pairIdx1Choice);
pairIdx2 = idxs2(pairIdx2Choice);
label = 0;
end
%***************************************************************************************
function [X1,X2,pairLabels] = getAlexnetBatch(imds,miniBatchSize)
pairLabels = zeros(1,miniBatchSize);
X1 = zeros([227 227 3 miniBatchSize]);
X2 = zeros([227 227 3 miniBatchSize]);
imageAugmenter = imageDataAugmenter('RandRotation',[90,270],'RandXReflection',true,'RandYReflection',true);
imdsaug = augmentedImageDatastore([227 227],imds,'DataAugmentation',imageAugmenter);
batch=readall(imdsaug);
for i = 1:miniBatchSize
choice = rand(1);
if choice < 0.5
[pairIdx1,pairIdx2,pairLabels(i)] = getSimilarPair(batch.response);
else
[pairIdx1,pairIdx2,pairLabels(i)] = getDissimilarPair(batch.response);
end
X1(:,:,:,i) =batch.input{pairIdx1};
X2(:,:,:,i) =batch.input{pairIdx2};
end
end
function [pairIdx1,pairIdx2,pairLabel] = getSimilarPair(classLabel)
% getSimilarSiamesePair returns a random pair of indices for images
% that are in the same class and the similar pair label = 1.
% Find all unique classes.
classes = unique(classLabel);
% Choose a class randomly which will be used to get a similar pair.
classChoice = randi(numel(classes));
% Find the indices of all the observations from the chosen class.
idxs = find(classLabel==classes(classChoice));
% Randomly choose two different images from the chosen class.
pairIdxChoice = randperm(numel(idxs),2);
pairIdx1 = idxs(pairIdxChoice(1));
pairIdx2 = idxs(pairIdxChoice(2));
pairLabel = 1;
end
function [pairIdx1,pairIdx2,label] = getDissimilarPair(classLabel)
% getDissimilarSiamesePair returns a random pair of indices for images
% that are in different classes and the dissimilar pair label = 0.
% Find all unique classes.
classes = unique(classLabel);
% Choose two different classes randomly which will be used to get a dissimilar pair.
classesChoice = randperm(numel(classes),2);
% Find the indices of all the observations from the first and second classes.
idxs1 = find(classLabel==classes(classesChoice(1)));
idxs2 = find(classLabel==classes(classesChoice(2)));
% Randomly choose one image from each class.
pairIdx1Choice = randi(numel(idxs1));
pairIdx2Choice = randi(numel(idxs2));
pairIdx1 = idxs1(pairIdx1Choice);
pairIdx2 = idxs2(pairIdx2Choice);
label = 0;
end
Richard
Richard 2022-3-24
@Cloud Wind thanks for the code, I think this will be very helpful for us in understanding the exact nature of the issue.
For now, the best suggestion I have is to continue using Acceleration="none".

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Richard
Richard 2022-3-24
The errot itself is related to performance optimizations within the dlnetwork class. You should be able to prevent it by specifying ("Acceleration", "none") as an additional parameter-value pair when you call forward on the network, i,.e.:
Y = forward(net, X, "Acceleration", "none")
(Obviously this may also have an adverse impact on performance, unfortunately)

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Operations 的更多信息

标签

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by