Out of memory on device. To view more detail about available memory on the GPU, use 'gpuDevice()'. If the problem persists, reset the GPU by calling 'gpuDevice(1)

Question

john karli 2021-11-24

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1594854-out-of-memory-on-device-to-view-more-detail-about-available-memory-on-the-gpu-use-gpudevice-i

评论： yanqi liu 2021-11-25

I want to train the model in .mat dataset but i am getting the memory error my dataset size is [256,340,2] when i try

 gpuDevice(1)
ans = 
  CUDADevice with properties:
                      Name: 'NVIDIA GeForce GTX 1080 Ti'
                     Index: 1
         ComputeCapability: '6.1'
            SupportsDouble: 1
             DriverVersion: 11.4000
            ToolkitVersion: 11
        MaxThreadsPerBlock: 1024
          MaxShmemPerBlock: 49152
        MaxThreadBlockSize: [1024 1024 64]
               MaxGridSize: [2.1475e+09 65535 65535]
                 SIMDWidth: 32
               TotalMemory: 1.1811e+10
           AvailableMemory: 1.0615e+10
       MultiprocessorCount: 28
              ClockRateKHz: 1620000
               ComputeMode: 'Default'
      GPUOverlapsTransfers: 1
    KernelExecutionTimeout: 1
          CanMapHostMemory: 1
           DeviceSupported: 1
           DeviceAvailable: 1
            DeviceSelected: 1

code is

            location = 'D:\data-11\sir task\dataset\';
imds = imageDatastore(location, 'FileExtensions', '.mat', 'IncludeSubfolders',1, ...
                      'LabelSource','foldernames',...
                      'ReadFcn',@matReader);
[imdsTrain,imdsValidation] = splitEachLabel(imds,0.7, 'randomized');
net = lgraph_1;
inputSize = lgraph_1.Layers(1).InputSize;
[learnableLayer,classLayer] = findLayersToReplace(lgraph_1);
[learnableLayer,classLayer] 
numClasses = numel(categories(imdsTrain.Labels));
if isa(learnableLayer,'nnet.cnn.layer.FullyConnectedLayer')
    newLearnableLayer = fullyConnectedLayer(numClasses, ...
        'Name','new_fc', ...
        'WeightLearnRateFactor',10, ...
        'BiasLearnRateFactor',10);
    
elseif isa(learnableLayer,'nnet.cnn.layer.Convolution2DLayer')
    newLearnableLayer = convolution2dLayer(1,numClasses, ...
        'Name','new_conv', ...
        'WeightLearnRateFactor',10, ...
        'BiasLearnRateFactor',10);
end
lgraph_1 = replaceLayer(lgraph_1,learnableLayer.Name,newLearnableLayer);
newClassLayer = classificationLayer('Name','new_classoutput');
lgraph_1 = replaceLayer(lgraph_1,classLayer.Name,newClassLayer);
miniBatchSize = 128;
valFrequency = floor(numel(imdsTrain.Files)/miniBatchSize);
checkpointPath = pwd;
options = trainingOptions('sgdm', ...
    'MiniBatchSize',miniBatchSize, ...
    'MaxEpochs',100, ...
    'InitialLearnRate',1e-3, ...
    'Shuffle','every-epoch', ...
    'ValidationData',imdsValidation, ...
    'ValidationFrequency',valFrequency, ...
    'Verbose',false, ...
    'Plots','training-progress', ...
    'CheckpointPath',checkpointPath);
net = trainNetwork(imdsTrain,lgraph_1,options);