Speeding up the nested for loops

2022 11 4
0 个回答
更新时间：2022 11 8
7 次查看（30 天）

0 个投票

Hi,
I have a nested for loop that inside the innermost loop, the result of each iteration should be saved in a 4D matrix.
Althought the calculation is more accurate than curve fitting but the speed is awful. How can I speed up this piece of code? Parloop fails due to the way I save the data.

9 个评论
显示 7更早的评论 隐藏 7更早的评论

KSSV
KSSV 2022-11-4
Things to consider:
  1. Is really loop required?
  2. Can't it be vectorsed?
  3. Did you initialize the arrays filling in loop?
" Did you initialize the arrays filling in loop? "
Yes!
" Can't it be vectorsed? "
I dunno how it is possible in this problem.
Raymond Norris
Raymond Norris 2022-11-4
I'm puzzled how DistChoiceAllParams isn't overwritten by the last iteration of N? That is, you run line 99 SampN times (i.e., 1000). No matter what the value of DistChoiceAllParams is after the 1st 999 iterations, the last time through the Murange loop is what will be assigned to DistChoiceAllParams, no?
Unless you're doing something on line 109 (e.g., plotting DistChoiceAllParams or writing DistChoiceAllParams to a file), I don't see the purpose of the outer for-loop.
The reason is I want to span a 5 dimensional space and find the minimum of this space. It is important to know the index of each of dimesions. That's why I saved all indexes. for N = 1 : SampN , I repeat all the calculations with different shuffling and resampling of my data.
Anyway I chose another way to do this calculation. I created a 2D matrix with 600k elements as my space (the same number of elements but squezeed in 2D) and I span this 2D matrix instead of 5D. With this 2D matrix and with the use of ''par for'' still the consumption of time is awful.
Santosh Fatale
Santosh Fatale 2022-11-8
Hi Zahra,
Could you share the sample code?
Marcel
Marcel 2022-11-8
By having so many for loops, its not surprising me that its being slow, just from first sight. Without knowing what the code actually does we cant help you in investigating on what might take a lot of resources/time to execute.
Zahra Yousefi Darani
编辑：Bruno Luong 2022-11-8
Thanks a lot. Here it is the code:
close all, clear,clc
%%
tic
SampN = 1000
Params = zeros(SampN,4);
subjects = [7,8,10,11];
for SUBJ = 1 : 4
SubjectNumber = subjects(SUBJ)
dataname = sprintf('FB2ZY%d',SubjectNumber)
data = cell2mat(struct2cell(load(dataname)))
stims = data(:,3);
stim = unique(stims)
stimbound = stim([5,6]);
data(:,7) = data(:,3);
for bb = 1 : numel(stimbound)
filter = data(:,3)==stimbound(bb);
data(filter,7) = unique(data(:,2));
end
clear stims
clear stim
stims = data(:,7);
stim = unique(stims);
distance = mean(diff(stim))
for sb = 1 : numel(stims)
data(sb,8) = round((data(sb,7)-data(sb,2))/distance);
end
choices = data(:,6);
clear stims
clear stim
stims = data(:,8);
stim = unique(stims);
IDs = stim;
meanChoice = nan(numel(IDs),1); % calculation of y data
numChoice = nan(numel(IDs),1); %size of y data
for iID = 1:numel(IDs)
meanChoice(iID) = mean(choices(stims==IDs(iID)));
numChoice(iID) = sum(stims==IDs(iID));
end
%% From here I want to optimize the code, above part runs in a few portion of second.
MaxSigma = (numel(stim)-1)*distance;
Maxlapse = 0.1;
SigmaRange = 0 : 0.1 : 3;
LapseRange = 0 : 0.01 : Maxlapse;
MinMu = median(stim)-1;
MaxMu = median(stim)+1;
Murange = MinMu : 0.1 : MaxMu;
%%
DistChoiceAllParams = zeros(numel(LapseRange),numel(LapseRange),numel(SigmaRange),numel(Murange));
calchoice = zeros(SampN,750);
choice = choices;
SessID = find(data(:,1) == 1);
for N = 1 : SampN
SampleID = randsample([1:numel(SessID)],numel(SessID),true);
SampleStim = data(SessID(SampleID(1)):SessID(SampleID(1))+249,8);
SampleChoice = data(SessID(SampleID(1)):SessID(SampleID(1))+249,6);
for ss = 2 : numel(SampleID)
SampleStim = vertcat(SampleStim,data(SessID(SampleID(ss)):SessID(SampleID(ss))+249,8));
SampleChoice = vertcat(SampleChoice,data(SessID(SampleID(ss)):SessID(SampleID(ss))+249,6));
end
NeededStim(N,:) = SampleStim;
NeededChoice(N,:) = SampleChoice;
end
for N = 1 : SampN
S = NeededStim(N,:);
C = NeededChoice(N,:);
for m = 1 : numel(Murange)
mu = Murange(m);
for sig = 1 : numel(SigmaRange)
sigma = SigmaRange(sig);
for la = 1 : numel(LapseRange)
lambda = LapseRange(la);
for ga = 1 : numel(LapseRange)
gamma = LapseRange(ga);
calchoice(N,:) = CalChoice(gamma,lambda,S,mu,sigma);
DistChoiceAllParams(ga,la,sig,m) = CrossEntr(C,calchoice(N,:));
end
end
end
end
[MIN,idx] = min(DistChoiceAllParams,[],'all');
[I1,I2,I3,I4] = ind2sub(size(DistChoiceAllParams),idx);
Params(N,:) = [Murange(I4),SigmaRange(I3),LapseRange(I2),LapseRange(I1)];
end
name = strcat('Params_',dataname);
feval('assignin','base',name,Params)
save(name,name)
end
Jan
Jan 2022-11-8
编辑：Jan 2022-11-8
We cannot run this code due to the missing input files. Then it is very hard to improve code, which is written as a large monolithic block, because we have to guess, where the bottleneck is.
Split the code into functions. Use a function as main part also instead of using the brute clearing header close all, clear,clc. Avoid clear of variabels, because this is usually (but not in all cases) a waste of time in Matlab.
Avoid the izterative growing of arrays, because this is very expensive:
NeededStim = zeros(SampN, ???); % Pre-allocate accordingly!!!
NeededChoice = zeros(SampN, ???); % Pre-allocate accordingly!!!
for N = 1 : SampN
SampleID = randsample([1:numel(SessID)],numel(SessID),true);
SampleStim = zeros(numel(SampleID), 250);
SampleChoice = zeros(numel(SampleID), 250);
for ss = 1 : numel(SampleID)
SampleStim(:, ss) = data(SessID(SampleID(ss)):SessID(SampleID(ss))+249,8);
SampleChoice(:, ss) = data(SessID(SampleID(ss)):SessID(SampleID(ss))+249,6);
end
NeededStim(N,:) = SampleStim(:);
NeededChoice(N,:) = SampleChoice(:);
end
This is ugly:
feval('assignin','base',name,Params)
Because this is a script, you create a variable dynamically in the local workspace. As eval this impedes Matlab's JIT acceleration drastically: Afterwards Matlab has to check the look-up table of variables instead of using efficient pointers to the values. This can slow down loops by a factor of 100. "Optimizing" this code is not the point inthis case, but cleaing it from evil pitfalls.
Oh! Thanks for very constructive comments. I am going to rewrite the cod.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Loops and Conditional Statements 的更多信息

产品

版本

R2022a

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by