I have a calculation to do on a large list of points, that makes use of two large cell arrays (~1.5 GB). My efforts in changing the main loop of the code from for to parfor seems to slow it down by a factor of the number of workers (all on a local machine).
The code I am trying to run looks like:
Main body:
idx_vals = {cell array of size n};
W_vals = {cell array of size n};
idx = parallel.pool.Constant(idx_vals);
W = parallel.pool.Constant(W_vals);
out_val = func(idx, W);
Function call:
function val = func(idx, W);
end
n = length(idx);
val = zeros(1, n);
parfor k=1:n
idx_k = idx.Value(k);
idx_k = idx_k{1};
W_k = W.Value(k);
W_k = W_k{1};
val(k) = [some function of idx_k and W_k];
end
Some possibly useful comments:
- idk and W are cell arrays containing matrices as elements. I am using a cell array because the size of the matrix changes from element to element.
- I also looked at making idx and W global variables, but the compiler gave warnings about that being a bad idea.
- I also tried versions of the code where idx and W were global variables instead of parallel.pool.Constants.
In everything I am trying, it seems that the transfer or access of the large variables idk and W dominates the run time, and that run time is roughly the number of workers * the run time for a for loop instead of parfor.
Any ideas of what I am doing wrong or what else to try?
