Using parfeval for sequential calculations
6 次查看(过去 30 天)
显示 更早的评论
In the Matlab documentation, all of the examples for parfeval function use a similar implementation, where the full set of jobs are submitted at the same time, within a single for loop. These examples include a clever strategy, where the job with the largest number is submitted first, and successive jobs are submitted in decreasing order ending with the "first" job: F(numCores) --> F(1). Check Parfeval Blackjack for an example of this strategy: https://www.mathworks.com/help/distcomp/examples/parfeval-blackjack.html. This trick accounts for the fact that there is no way to initialize the FevalFuture object, which is the output argument of the parfeval function. I would expect a function like gobjects, which can be used to initialize a set of graphics objects, but there is nothing like this for the FevalFuture object.
This limitation makes it difficult to use parfeval in a sequential fashion. My application is a search algorithm that finds the minimium point for a nonlinear function. In the pseudocode example below, the function to be minized is called fObject, and it has variables (input arguments) represented by the vector x. The goal is find the vector x that minimizes fObject. The search has a function fGenerate that selects candidate solution, xCan, using information from previous candidate solutions. The function fEvaluate determines if and when the search has found the global minimum.
This problem requires sequential submission of jobs via parfeval. My code below works, but it kludgy and confusing. Let me highlight the issues:
- The FevalFuture object is initialized by a call using a nonsense function (@null) and by defining the output to F(numCore), where numCores is the maximum number of available cores.
- The loop evaluates completed jobs and then generates a new candidate variable point for the search. There are no active jobs at the start of the search, so iCore is used to count downward from numCores to 0. The first numCores loops are used to start jobs on the available cores. The jobs are started at the end of the loop, so the iCore==0 testis used to determine when this initial set of jobs has been submitted.
- After this initial phase, each successive job is assigned to the core that gave the last solution (iCan). At this point, everything is simple and clean.
My wish list follows:
- I am hoping for a better approach, with a specific example, that illustrates a best-practices approach for this sequential application.
- A function, perhaps named fevalObjects (like gobjects), that would initialize a FevalFuture object.
- A default option for Parfeval for assigning jobs to F to objects that are unassigned or previously. At present, one has to specify the index in F for each submitted job.
I am hoping that there are others who have given these issue more thought and have devised clever solutions. My google searching indicates that there is nothing yet reported on the web.
PSEUDOCODE EXAMPLE
%... Setup pool of cores
p = gcp;
numCores = p.NumWorkers;
%... Initialize F as parallel.FevalFuture object, using a nonsense function (@null)
iCore = numCores;
F(numCores) = parfeval(p, @null, 0, 0);
while true
if iCore==0
%... Fetch a completed job
[iCan, fCan] = fetchNext(fJob);
xCan = cell2mat(fJob(iCan).InputArguments);
%... Evaluate job and check stop criteria to terminate search
if fEvaluate(fCan, fOthers) < tolerance, break, end
end
%... Generate new candidate point for next step of the search
xCan = fGenerate(...);
%... Submit job for new candidate point
if iCore>0, iCan = iCore; end
F(iCan) = parfeval(p, fObject, 1, xCan);
if iCore>0, iCore = iCore - 1; end
end
0 个评论
采纳的回答
Edric Ellis
2017-2-16
As you observe, there is no way to pre-allocate arrays of FevalFuture objects. In this case, I would simply use cell arrays instead - yes, there is some extra overhead here, but it's probably trivial compared to the remote invocation cost. Here's roughly how I might approach this:
p = gcp();
N = p.NumWorkers;
fCell = {};
tolerance = 1e-5;
numSubmitted = 0;
while true
if numel(fCell) == N
[idx, cost] = fetchNext([fCell{:}]);
x = fCell{idx}.InputArguments{1};
fCell(idx) = [];
if cost < tolerance
result = x;
break;
end
end
% Generate a new search point
x = rand();
% Cost function evaluation
fCell{end+1} = parfeval(p, @(val) val.^2, 1, x);
end
cancel([fCell{:}]);
更多回答(1 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Asynchronous Parallel Programming 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!