In the Matlab documentation, all of the examples for parfeval function use a similar implementation, where the full set of jobs are submitted at the same time, within a single for loop. These examples include a clever strategy, where the job with the largest number is submitted first, and successive jobs are submitted in decreasing order ending with the "first" job: F(numCores) --> F(1). Check Parfeval Blackjack for an example of this strategy: https://www.mathworks.com/help/distcomp/examples/parfeval-blackjack.html. This trick accounts for the fact that there is no way to initialize the FevalFuture object, which is the output argument of the parfeval function. I would expect a function like gobjects, which can be used to initialize a set of graphics objects, but there is nothing like this for the FevalFuture object.
This limitation makes it difficult to use parfeval in a sequential fashion. My application is a search algorithm that finds the minimium point for a nonlinear function. In the pseudocode example below, the function to be minized is called fObject, and it has variables (input arguments) represented by the vector x. The goal is find the vector x that minimizes fObject. The search has a function fGenerate that selects candidate solution, xCan, using information from previous candidate solutions. The function fEvaluate determines if and when the search has found the global minimum.
This problem requires sequential submission of jobs via parfeval. My code below works, but it kludgy and confusing. Let me highlight the issues:
- The FevalFuture object is initialized by a call using a nonsense function (@null) and by defining the output to F(numCore), where numCores is the maximum number of available cores.
- The loop evaluates completed jobs and then generates a new candidate variable point for the search. There are no active jobs at the start of the search, so iCore is used to count downward from numCores to 0. The first numCores loops are used to start jobs on the available cores. The jobs are started at the end of the loop, so the iCore==0 testis used to determine when this initial set of jobs has been submitted.
- After this initial phase, each successive job is assigned to the core that gave the last solution (iCan). At this point, everything is simple and clean.
My wish list follows:
- I am hoping for a better approach, with a specific example, that illustrates a best-practices approach for this sequential application.
- A function, perhaps named fevalObjects (like gobjects), that would initialize a FevalFuture object.
- A default option for Parfeval for assigning jobs to F to objects that are unassigned or previously. At present, one has to specify the index in F for each submitted job.
I am hoping that there are others who have given these issue more thought and have devised clever solutions. My google searching indicates that there is nothing yet reported on the web.
PSEUDOCODE EXAMPLE
p = gcp;
numCores = p.NumWorkers;
iCore = numCores;
F(numCores) = parfeval(p, @null, 0, 0);
while true
if iCore==0
[iCan, fCan] = fetchNext(fJob);
xCan = cell2mat(fJob(iCan).InputArguments);
if fEvaluate(fCan, fOthers) < tolerance, break, end
end
xCan = fGenerate(...
if iCore>0, iCan = iCore; end
F(iCan) = parfeval(p, fObject, 1, xCan);
if iCore>0, iCore = iCore - 1; end
end
