Except for the first iteration others show NaN

Question

KAI-YANG WANG 2021-8-31

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1444264-except-for-the-first-iteration-others-show-nan

评论： KAI-YANG WANG 2021-9-2

When I start training my CNN it shows me training details only on first iteration, and I've already check the training data size and the validation data size is 100000 and 20000, I was wondering is it normal process or not. Thank you.

2 个评论
显示无隐藏无

Chunru 2021-9-1

Check if the train data contains any nan.

KAI-YANG WANG 2021-9-1

Thank you for the answer, Ive checked there's no nan in my training data.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

KSSV 2021-9-1

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1444264-except-for-the-first-iteration-others-show-nan#answer_778334

在 MATLAB Online 中打开

Try reducing the initial learning rate.

options = trainingOptions('adam') ;
options.InitialLearnRate
ans = 1.0000e-03
options.InitialLearnRate = 10^-4 ;
options.InitialLearnRate
ans = 1.0000e-04

If the above works well and good. If not show us your full code and data.

2 个评论
显示无隐藏无

KAI-YANG WANG 2021-9-1

在 MATLAB Online 中打开

Thank you so much and I've try to reduce the initial learning rate, but still the same. My code is attached below.

location='\\ikb\home\25\1939725\Desktop\cost2100-master mimo\matlab\CNN\csitrain\Htrain'
imds = imageDatastore(location,'FileExtensions','.mat','IncludeSubfolders',1, ...
                      'LabelSource','foldernames');
imdsx=imageDatastore(location,'FileExtensions','.mat','IncludeSubfolders',1, ...
                      'LabelSource','foldernames');
imdsy=imageDatastore(location,'FileExtensions','.mat','IncludeSubfolders',1, ...
                      'LabelSource','foldernames');
                  
countEachLabel(imds)
trainingDS = imds;
trainingDS.Labels = categorical(trainingDS.Labels);
trainingDS.ReadFcn=@readFcn1;
countEachLabel(imdsx)
trainingDS2 = imdsx;
trainingDS2.Labels = categorical(trainingDS.Labels);
trainingDS2.ReadFcn=@readFcn2;
countEachLabel(imdsy)
trainingDS2 = imdsy;
trainingDS2.Labels = categorical(trainingDS.Labels);
trainingDS2.ReadFcn=@readFcn3;
dsTrain = transform(imds,@commonPreprocessing);
dsVal = transform(imdsx,@commonPreprocessing);
dstest = transform(imdsy,@commonPreprocessing);
trainData=combine(dsTrain,dsTrain);
valData=combine(dsVal,dsVal);
%Designed concolutional network
maxEpochs = 200;
options = trainingOptions('sgdm', ...
    'ValidationData', valData,...
    'InitialLearnRate',0.0001,...
    'ExecutionEnvironment','gpu', ...
    'LearnRateSchedule','piecewise', ...
    'MaxEpochs',maxEpochs, ...
    'MiniBatchSize',200, ...
    'Plots','training-progress')
doTraining = true; 
if doTraining  
    modelDateTime = string(datetime('now','Format',"yyyy-MM-dd-HH-mm-ss"));
    net =trainNetwork(trainData, lgraph, options);
    save(strcat("trainedCNN-",modelDateTime,"-Epoch-",num2str(maxEpochs),".mat"),'net');
else 
    load('traindata.mat'); 
end
YPredicted = predict(net,dstest);
function dataOut = commonPreprocessing(data)
    dataOut = cell(size(data));
    for col = 1:size(data,2)
        for idx = 1:size(data,1)
            temp = single(data{idx,col});
            temp = reshape(temp,[32,32,2]);
            temp = rescale(temp);
            dataOut{idx,col} = temp;
        end
    end
end