problem with parfor and struct
显示 更早的评论
Hey guys,
i have some problems with my parfor-loop. To explain my programm, i have written down an less complex version of my programm, wich is shown below.
I have two structures with some fields in it, wich i need for calcutations. So the idear is to calculate, struct1(1) with strucht2(1 to n), strucht1(2) with struct2(1 to n), ..., struct(m) with struct2(1 to n). My problem is, that the programm is slower with parfor then with for. I found out, that the problem is the line with "output(:, k) = output_z; ", but i dont know how to solve it different, because otherwise i dont get the output out of the parfor-loop.
If you have an idear what i could do, even if it something totaly different, im happy for your aswer.
Kind regards,
Daniel
struct1(200)=struct();
for k = 1:200
struct1(k).constant.A = round(5 + rand(1)*10);
struct1(k).constant.B = round(2 + rand(1)*10);
end
struct2(500)=struct();
for a = 1 : 500
struct2(a).d = 1+rand(1);
end
tic
parfor k = 1 : 200
[output_z] = loop(struct1(k), struct2)
output(:, k) = output_z;
end
toc
function [output] = loop(struct1, struct2)
for a = 1 : 500
[output(a)] = calculation(struct1, struct2(a));
end
end
function [struct1] = calculation(struct1, struct2)
struct1.pv = struct1.constant.A + struct1.constant.B * struct2.d;
end
9 个评论
Walter Roberson
2019-5-6
Do not grow variables dynamically. Pre-allocate output. Even if what it takes is
a = 1;
output(a) = calculation(struct1, struct2(a));
output(500).pv = []; %extends output to full size
for a = 2 : 500
output(a) = calculation(struct1, struct2(a));
end
In the case where the iterations are unrelated, as appears to be the case here, you can take the shortcut of instead programming
for a = 500 : -1 : 1
output(a) = calculation(struct1, struct2(a));
end
this creates the last entry and moves towards the first, and therefore never needs to make the array longer.
Daniel Ermer
2019-5-6
Bjorn Gustavsson
2019-5-6
If your problem is this trivially parallelizable and it is imperative that you get a significant speedup to get the job done perhaps you can parallelize it at the matlab-process-level with standard loops? It is an ugly suggestion - but if no calculation depends on the other results it ought to be possible to run batches of matlab-sessions starting different processing-runs with the -r trick:
> matlab -nodesktop -r process_0to50
> matlab -nodesktop -r process_51to100
...and so on. It is not pretty, but might get the job done, if doing it that way I'd start the processes in screen-sessions so that I can hide them from sight too...
Daniel Ermer
2019-5-7
Matt J
2019-5-7
What sort of parallel resources are you running this on? Not a cluster, I am guessing.
Bjorn Gustavsson
2019-5-7
@Daniel Ermer: I figured that it would be an "unsuitable" solution.
Do you absolutely need the structs? Can you re-organize your code using more basic (matrices...) data-types? If that wouldn't comlpetely kill the code-structure maybe you give parfor an easier time to figure out what memory is and is not affected by the calculations? <speculation>Once uppon a time when structs were new I managed to write a functions using structs with .next and .prev "pointing" to similar structs as if they were c-style pointers - even though that might not have been the case. I don't know if a struct-field pointing to another struct would now behave as a pointer, but if it did it would be possible to redirect it from its initial value in a function. If parfor is programmed to be cautious to such events then that might be why you get your problems...<\speculation>
Daniel Ermer
2019-5-7
Walter Roberson
2019-5-7
It is quite common for parallel programs to be slower than serial.
- overhead of sending data back and forth, especially if it decides that a variable has to be a broadcast variable instead of sliced
- Parallel programs by default only get one thread per worker. If the work being done happens to be match one of the patterns common in linear algebra type operations (including just adding two matrices), then for large enough matrices, MATLAB normally calls one of the high performance multithreaded libraries. With workers only getting one thread, those end up being done in serial, with the difference being that several of those might be happening at the same time. It is common that you end up getting higher performance leaving the routines non-parallel
- parallel typically wins for cases where not much memory needs to be shared and where the operations are quite independent and where the operations do not paralellize well. That happens often enough to be useful, but parfor taking 1.5 to 5 times longer is pretty common.
Daniel Ermer
2019-5-8
回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Loops and Conditional Statements 的更多信息
2019-5-6
重新打开：
2019-5-13Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
