Keep parpool open instead of open/close for each iteration. Possible?

11 次查看（过去 30 天）
Sebastian Sas Brunser
评论： Mario Malic 2024-1-16
Good day and thank you in advance for your time and help. I am running a surrogate optimization using the surrogateopt script, that runs a ray tracer which uses a parpool to trace the rays.
An exemplary code of the ray tracer would be the following:
parpool('local',4);
Number_of_rays = 100000;
spmd
for i=drange(1:Number_of_rays)
% ray's are traced
%if hit, a counter is raised
if hit==true
absorption_counter=absorption_counter+1;
end
end
labBarrier % we wait for all the workers to stop tracing
absorption_counter = gplus(absorption_counter); %we add the counters of each worker
end %spmd
delete(gcp('nocreate'))
So surrogateopt call the ray tracer and run it many times. However, I've noticed that the time that takes to start the parallel pool is quite high.
Starting parallel pool (parpool) using the 'local' profile ...
I was just wondering if you can somehow keep the pool open and run the optimization iteratively but opening the parpool once, and then not closing it.
I guess that the absorption_counter would need to be resetted each time (set to 0 before running again), but I don't know if this would work.
Thank you for your input!
Best,
Sebastian

请先登录，再回答此问题。

回答（2 个）

Hassaan
Hassaan 2024-1-5
编辑：Hassaan 2024-1-5
% Initialize the parallel pool with the desired number of workers
% Ensure this number does not exceed the maximum number of workers allowed on your system
poolObj = gcp('nocreate'); % Get the current parallel pool object
if isempty(poolObj)
poolObj = parpool('local', 2); % Adjust the number of workers according to your system
end
% Define the number of rays and iterations
Number_of_rays = 100000;
number_iterations = 50; % Set the number of iterations for the secant method
% Preallocate the matrix to store iteration results
results_matrix = zeros(number_iterations, 3);
% Main loop for the secant method
for iter = 1:number_iterations
% Initialize the absorption counter for this iteration
absorption_counter = 0;
% Run the computation in parallel
spmd
local_absorption_counter = 0; % Initialize local counter for each worker
% Parallel loop for rays tracing
for i = drange(1:Number_of_rays)
% Simulate ray tracing here...
hit = rand < 0.5; % Dummy condition for hit
% If hit, increment the local counter
if hit
local_absorption_counter = local_absorption_counter + 1;
end
end
% Synchronize the workers and sum the local counters
spmdBarrier;
absorption_counter = spmdPlus(local_absorption_counter);
end
% Collect the result from the parallel block (spmd)
total_absorption = absorption_counter{1}; % Get the result from the first lab
% Calculate the error (placeholder values for exact and approx)
% You would replace 'exact_solution' with the value of your exact solution
exact_solution = 12345; % Placeholder for the exact solution
approx_solution = total_absorption; % Placeholder for the approximated solution from the current iteration
error = abs((exact_solution - approx_solution) / exact_solution) * 100;
% Store the iteration results
results_matrix(iter, :) = [iter, approx_solution, error];
end
% Display the results matrix
disp('Iteration results:');
disp(results_matrix);
% Close the parallel pool if you're done with all parallel computations
% delete(poolObj); % Uncomment this line if you want to close the pool
Please replace the placeholders like exact_solution, approx_solution, and the dummy condition for hit with your actual logic and calculations.
Note:
I used MATLAB R2023b
Output:
Iteration results:
1.0e+04 *
0.0001 4.9976 0.0305
0.0002 4.9877 0.0304
0.0003 5.0095 0.0306
0.0004 4.9981 0.0305
0.0005 4.9823 0.0304
0.0006 4.9939 0.0305
0.0007 4.9803 0.0303
0.0008 5.0107 0.0306
0.0009 4.9869 0.0304
0.0010 4.9901 0.0304
0.0011 5.0248 0.0307
0.0012 5.0106 0.0306
0.0013 5.0295 0.0307
0.0014 4.9952 0.0305
0.0015 5.0119 0.0306
0.0016 4.9926 0.0304
0.0017 4.9979 0.0305
0.0018 4.9810 0.0303
0.0019 5.0103 0.0306
0.0020 4.9841 0.0304
0.0021 5.0484 0.0309
0.0022 5.0156 0.0306
0.0023 4.9970 0.0305
0.0024 4.9889 0.0304
0.0025 4.9937 0.0305
0.0026 5.0417 0.0308
0.0027 4.9775 0.0303
0.0028 4.9955 0.0305
0.0029 5.0080 0.0306
0.0030 4.9746 0.0303
0.0031 5.0127 0.0306
0.0032 4.9700 0.0303
0.0033 5.0011 0.0305
0.0034 4.9860 0.0304
0.0035 5.0043 0.0305
0.0036 5.0167 0.0306
0.0037 4.9996 0.0305
0.0038 4.9999 0.0305
0.0039 4.9877 0.0304
0.0040 4.9733 0.0303
0.0041 4.9969 0.0305
0.0042 4.9934 0.0304
0.0043 4.9995 0.0305
0.0044 5.0303 0.0307
0.0045 5.0180 0.0306
0.0046 4.9946 0.0305
0.0047 5.0026 0.0305
0.0048 4.9867 0.0304
0.0049 5.0093 0.0306
0.0050 4.9710 0.0303
------------------------------------------------------------------------------------------------------------------------------------------------
If you find the solution helpful and it resolves your issue, it would be greatly appreciated if you could accept the answer. Also, leaving an upvote and a comment are also wonderful ways to provide feedback.
Professional Interests
  • Technical Services and Consulting
  • Embedded Systems | Firmware Developement | Simulations
  • Electrical and Electronics Engineering
  4 个评论
显示 2更早的评论
Sebastian Sas Brunser
Sebastian Sas Brunser 2024-1-12
Hi Mario,
Yes, true. But you can't run parallel job on parlallel optimization. It doesn't branch, so I would need to change the code. Anyways, I don't know if changing which part is parallelize will solve the problem. But yes, thanks!
Mario Malic
Mario Malic 2024-1-16
When you say, it doesn't branch - do you expect parallel in that case means that each worker should have an initial guess and start optimization process separately? If that's what you want, this is an option https://uk.mathworks.com/help/gads/multistart.html

请先登录，再进行评论。

Hassaan
Hassaan 2024-1-5
@Mario Malic A basic approach. Let me know if it helps. Thanks.
% Initialize the parallel pool
poolObj = gcp('nocreate'); % If no pool, do not create new one.
if isempty(poolObj)
poolObj = parpool('local', 2); % Modify based on your system's capability
end
% Set the number of simulations and other parameters
num_simulations = 10000;
% ... (Other parameters initialization)
% Define the surrogate optimization function
opts = optimoptions('surrogateopt', 'UseParallel', true);
% Define the objective function for surrogate optimization
objectiveFunction = @(x) myObjectiveFunction(x); % Define your objective function properly
% Call the surrogate optimization
[sol, fval, exitflag, output] = surrogateopt(objectiveFunction, lb, ub, opts);
% After the optimization
% ... (Do something with the results)
% Optionally close the parallel pool if you're done with all parallel computations
% delete(gcp('nocreate'));
  • The parallel pool is checked and created if not already initialized.
  • The optimization options are set to use the parallel pool.
  • The objective function is defined, which should include the logic for your specific optimization problem.
  • surrogateopt is called with the objective function, lower and upper bounds (lb and ub), and the options structure.
  • After the optimization, you can process the results as needed.
You need to replace myObjectiveFunction with your actual objective function and define the lower and upper bounds (lb and ub) according to your problem.
The parallel pool will remain open throughout the surrogate optimization process. Once all computations are complete, you can choose to close the pool or leave it open if further parallel computations will follow. Remember that keeping the parallel pool open consumes system resources, so close it when it's no longer needed.
------------------------------------------------------------------------------------------------------------------------------------------------
If you find the solution helpful and it resolves your issue, it would be greatly appreciated if you could accept the answer. Also, leaving an upvote and a comment are also wonderful ways to provide feedback.
Professional Interests
  • Technical Services and Consulting
  • Embedded Systems | Firmware Developement | Simulations
  • Electrical and Electronics Engineering

类别

在 Help Center 和 File Exchange 中查找有关 Surrogate Optimization 的更多信息

标签

产品


版本

R2023b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by