I understand you are getting performance differences in the two workflows : using “parfor” (option-1) and running separate scripts(option-2). This difference can be attributed to how MATLAB handles parallel execution and resource allocation.
Some key factors because of which option-2 is performing better are as follows:
- “parfor” occurs some overhead due to parallel pool setup, data transfer and synchronization. This overhead can sometimes outweigh the benefits of parallel execution, especially for small or quick computations.
- All iterations in “parfor” share same MATLAB session, which can lead to contention for shared resources like memory and I/O.
- Option-2’s better performance can also be attributed to the fact that in this situation, the operating system can schedule each script to run on different cores, potentially leading to better utilization.
To achieve Option-2 performance in single script, you can use following strategies:
- Manual Parallel execution:
You can use system calls to invoke separate MATLAB sessions for each task. Following is the pseudo code for your reference:
system('matlab -batch "my_function(3)" &');
system('matlab -batch "my_function(7)" &');
system('matlab -batch "my_function(9)" &');
2. Batch Processing: you can use MATLAB’s batch function to run tasks in the background, which can help distribute tasks more evenly across cores. Following is the pseudo code for your reference:
jobs = [
batch(@my_function, 0, {3}),
batch(@my_function, 0, {7}),
batch(@my_function, 0, {9})
];
wait(jobs);
You can read more about "batch" function here: https://www.mathworks.com/help/parallel-computing/batch.html
You can read more about "system" function here: https://www.mathworks.com/help/matlab/ref/system.html
Hope this helps!
