pixelLabelImageDatastore not partitionable despite underlying imageDatastore and pixelLabelDatastore are

6 次查看（过去 30 天）
Jari Manni
Jari Manni 2022-2-18
回答： Joss Knight 2022-2-24
Hi,
I have a pixelLabelImageDatastore constructed with: pximds = pixelLabelImageDatastore(imds, pxds). Both imds and pxds are partitionable but the resulting pximds is not, which stops me from using multi-gpu.
To me it looks very strange. How do I make a partitionable pixelLabelImageDatastore?
>> isPartitionable(imds)
ans =
logical
1
>> isPartitionable(pxds)
ans =
logical
1
>> pximds = pixelLabelImageDatastore(imds, pxds);
>> isPartitionable(pximds)
ans =
logical
0

请先登录，再回答此问题。

采纳的回答

Joss Knight
Joss Knight 2022-2-24
For parallel training of a semantic segmentation network, you're supposed to use combine to join the two input datastores rather than using a pixelLabelImageDatastore.

更多回答（1 个）

yanqi liu
yanqi liu 2022-2-21
yes，sir，may be use partitionCamVidData，such as
https://ww2.mathworks.cn/help/releases/R2019a/vision/examples/semantic-segmentation-using-deep-learning.html
function [imdsTrain, imdsVal, imdsTest, pxdsTrain, pxdsVal, pxdsTest] = partitionCamVidData(imds,pxds)
% Partition CamVid data by randomly selecting 60% of the data for training. The
% rest is used for testing.
% Set initial random state for example reproducibility.
rng(0);
numFiles = numel(imds.Files);
shuffledIndices = randperm(numFiles);
% Use 60% of the images for training.
numTrain = round(0.60 * numFiles);
trainingIdx = shuffledIndices(1:numTrain);
% Use 20% of the images for validation
numVal = round(0.20 * numFiles);
valIdx = shuffledIndices(numTrain+1:numTrain+numVal);
% Use the rest for testing.
testIdx = shuffledIndices(numTrain+numVal+1:end);
% Create image datastores for training and test.
trainingImages = imds.Files(trainingIdx);
valImages = imds.Files(valIdx);
testImages = imds.Files(testIdx);
imdsTrain = imageDatastore(trainingImages);
imdsVal = imageDatastore(valImages);
imdsTest = imageDatastore(testImages);
% Extract class and label IDs info.
classes = pxds.ClassNames;
labelIDs = camvidPixelLabelIDs();
% Create pixel label datastores for training and test.
trainingLabels = pxds.Files(trainingIdx);
valLabels = pxds.Files(valIdx);
testLabels = pxds.Files(testIdx);
pxdsTrain = pixelLabelDatastore(trainingLabels, classes, labelIDs);
pxdsVal = pixelLabelDatastore(valLabels, classes, labelIDs);
pxdsTest = pixelLabelDatastore(testLabels, classes, labelIDs);
end
  1 个评论
Jari Manni
Jari Manni 2022-2-21
Hi, thanks for the answer. But I think this is a separate issue. My problem is not on the train set, val set, test set partitioning, but related to multi gpu training where training set is splited into different GPU cards.
When I specify 'ExecutionEnvironment', 'multi-gpu', I get the following error:
'Error using trainNetwork (line 184)
The input datastore is not Partitionable and does not support parallel operations.
UnderlyingCause: [1×1 MException]
identifier: 'nnet_cnn:cnn:GeneralDatastoreDispatcher:NonPartitionableDatastore'
message: 'The input datastore is not Partitionable and does not support parallel operations.'
cause: {}
stack: [2×1 struct]
Correction: []

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Parallel and Cloud 的更多信息

标签

产品


版本

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by