spmd inside parfor loop

8 次查看（过去 30 天）
Sankar Subbayya
Sankar Subbayya 2013-8-2
Description: I have two questions for running PCT in a local machine. Here two loops where the two loops runs and gives the right answers when running in a for loop; but runs into problems in a parfor loop.
(1) For the following loop, the code analyzer says that ,'PARFOR or SPMD cannot be used inside another PARFOR loop'
DataLength = size(MetaDataDS,1)
parfor IFile = 1:NoOfChannels
ChanDataDS = MetaDataDS;
FileName = strcat(SubDir,'\',FileNames{IFile});
ChanDataDS.DaysInPrint = ChanDataDaysInPrint(:,IFile);
ChanDataDS.Usage = ChanDataUsage(:,IFile);
a = char(ChannelNames{IFile});
[fidConfMat, fidRF, fidPlotError,fidPlotImp] = ...
createResultFilesTrees(ResultDirSub, a);
X = double(ChanDataDS(:,{'ORG_CONTENT_OWNER_ID', 'LIST_PRICE', ...
'PRIMARY_CATEGORY', 'TOTAL_PAGES','DaysInPrint'} ));
Y = nominal(ChanDataDS.Usage >0.0);
cv = cvpartition(DataLength,'holdout',holdoutPartition);
XTrain = X(cv.training, :);
XTest = X(cv.test,:);
YTrain = Y(cv.training);
tb = TreeBagger(nTrees, XTrain, YTrain, 'oobpred','on', ...
'NVarToSample','all','cat',[1,3], ...
'oobvarimp','on');
yp = predict(tb, XTest);
ConfMat = confusionmat(YTest, nominal(yp))
ObError = oobError(tb);
ObError(end)
PlotErrorImportant(ObError,fidPlotError,fidPlotImp, ...
tb.OOBPermutedVarDeltaError,char(ChannelNames{IFile}));
testResultSTemp = evalTreeModel(ConfMat,fidConfMat, ObError(end));
end
The same loop can run in simple for loop and gives the right 2x2 ConfMat (Confusion matrix). However, the same loop inside the parfor gives a ConfMat that has 1x1 ConfMat.
Any suggestions on how to avoid the code analyzer error, "PARFOR or SPMD cannot be used inside another PARFOR loop"
(2) In this case, I put the parfor outside another for loop. Here there is no code analyzer error. Here again the resulting Confmat is a wrong 1x1 ConfMat.
parfor Idir = 1:NoOfRootDirs
SubDir = strcat(currDataDir, '\',RootDirNames{Idir});
ResultDirSub = strcat(ResultDir, RootDirNames{Idir});
[logDir, confMatDir, plotDir, randForestDir] = ...
createResultsDirs (ResultDirSub);
ListOfFiles = dir(SubDir);
IsSub = ~[ListOfFiles(:).isdir]; %# returns logical vector
FileNames = {ListOfFiles(IsSub).name}';
NoOfChannels = numel(FileNames);
tic;
ChannelIdNum = zeros(NoOfChannels,1);
ChannelNames = cell(NoOfChannels,1);
ChanDataDaysInPrint = zeros(ndoc,NoOfChannels);
ChanDataUsage = zeros(ndoc,NoOfChannels);
for IFile = 1:NoOfChannels
ChannelId = FileNames{IFile}(1:strfind(FileNames{IFile},'.')-1);
ChannelIdNum(IFile) = str2double(ChannelId);
FileName = strcat(SubDir,'\',FileNames{IFile});
ChanData = importdata(FileName);
ChanData = sortrows(ChanData,1);
ChanDataUsage(:,IFile) = ChanData(:,2);
ChanDataDaysInPrint(:,IFile) = ChanData(:,3);
ChannelNames{IFile} = ChannelDS(ismember(ChannelDS.ORG_CHANNEL_ID, ...
ChannelIdNum(IFile)),:).ORG_CHANNEL_REFERENCE_NAME;
end
ChanDataDS = MetaDataDS;
% parfor IFile = 1:NoOfChannels
DataLength = size(MetaDataDS,1)
tic;
paroptions = statset('UseParallel',true);
for IFile = 1:NoOfChannels
ChanDataDS = MetaDataDS;
FileName = strcat(SubDir,'\',FileNames{IFile});
ChanDataDS.DaysInPrint = ChanDataDaysInPrint(:,IFile);
ChanDataDS.Usage = ChanDataUsage(:,IFile);
% ChanDataDS(ChanDataDS.LIST_PRICE>25000, :) =[];
% DataLength = size(ChanDataDS,1);
a = char(ChannelNames{IFile});
[fidConfMat, fidRF, fidPlotError,fidPlotImp] = ...
createResultFilesTrees(ResultDirSub, a);
X = double(ChanDataDS(:,{'ORG_CONTENT_OWNER_ID', 'LIST_PRICE', ...
'PRIMARY_CATEGORY', 'TOTAL_PAGES','DaysInPrint'} ));
Y = nominal(ChanDataDS.Usage >0.0);
cv = cvpartition(DataLength,'holdout',holdoutPartition);
XTrain = X(cv.training, :);
XTest = X(cv.test,:);
YTrain = Y(cv.training);
YTest = Y(cv.test);
% YTestN = nominal(YTest);
% YTrainN = nominal(YTrain);
% tic
tb = TreeBagger(nTrees, XTrain, YTrain, 'oobpred','on', ...
'NVarToSample','all','cat',[1,3], ...
'oobvarimp','on');
% toc
yp = predict(tb, XTest);
ConfMat = confusionmat(YTest, nominal(yp))
ObError = oobError(tb);
ObError(end)
PlotErrorImportant(ObError,fidPlotError,fidPlotImp, ...
tb.OOBPermutedVarDeltaError,char(ChannelNames{IFile}));
testResultSTemp = evalTreeModel(ConfMat,fidConfMat, ObError(end));
% testResultS{IFile} = testResultSTemp;
% fclose(fidConfMat);
% fclose(fidRF);
% fclose(fidPlotError);
% fclose(fidPlotImp);
end
end

请先登录，再回答此问题。

回答（1 个）

Edric Ellis
Edric Ellis 2013-8-6
All available parallelism is used up by the outermost PARFOR/SPMD, so there is no possible benefit to nesting these constructs. I would recommend reworking your code so that you do not nest them.
For your second case, I would recommend trying to narrow down the problem into a simple, self-contained example. Without that, it's hard to tell where the problem might lie. My guess would be that perhaps there is some setup phase missing on the workers. Remember that the body of the PARFOR loop is run in a separate MATLAB worker process - so it does not share any PERSISTENT or GLOBAL state.

类别

在 Help Center 和 File Exchange 中查找有关 Code Prover Analysis of AUTOSAR Code 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by