parallelizing optimization tasks for functions evaluated by external devices

Question

matlabUser 2011-10-11

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/17952-parallelizing-optimization-tasks-for-functions-evaluated-by-external-devices

Hello, I need to optimize two functions of different variables, let's call them F1(X1) and F2(X2), using GA algorithm. The functions are evaluated on an external device which allows for parallelization: you pass two variable sets {X1,X2} and get back two scalars {F1,F2} in just one step. A single step takes long time, so I am forced to do evaluations in parallel. I am trying to use Parallel Comp. Toolbox construct 'spmd' and having difficulty synchronizing function evaluations in the two labs running two different GA sessions. I had an idea to use 'gcat(X)' command to consolidate arguments {X1,X2} from different labs in order to pass them to external HW, but I don't know how to access arguments of GA and how to synchronize function calls from different labs. Any help ?? - Thank you.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Walter Roberson 2011-10-11

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/17952-parallelizing-optimization-tasks-for-functions-evaluated-by-external-devices#answer_24210

labSend() and labReceive() and labBarrier() perhaps?

3 个评论
显示 1更早的评论隐藏 1更早的评论

Walter Roberson 2011-10-11

spmd a chunk of code. The first thing the chunk of code checks is labindex. If it is 0, then make that invocation the mediator. Otherwise (other labs), call GA with a function handle, and in the function so designated, extract the current arguments and labSend them to lab 0, then labReceive to wait for the processed reply, and continue with the GA.

matlabUser 2011-10-11

OK, I wrote a couple of lines following your advise. Do you think the code below makes sense and is what you ment to say? I also missing if I need labBarrier or not, and if it is legal to use function definition inside spmd ... Thank you so much!!!

spmd(3)

if labindex==0

data1 = labReceive(1);

data2 = labreceive(2);

fullData = [data1,data2];

[result1 result2] = ProcessByExternalDevice(fullData);

labSend(result1,1); labSend(result2,2)

end

if labindex==1

% Create a function that extracts x1, sends to lab 0, and returns y1

function y1 = F1(x1)

% extracting argument x1:

labSend(x1,0);

y1 = labReceive(0);

end

% perform optimization:

X1out = ga(@F1, ...<parameters>... )

end

if labindex==2

function y2 = F2(x2)

% extracting argument x2:

labSend(x2,0);

y2 = labReceive(0);

end

% perform optimization:

X2out = ga(@F2, ...<parameters>... )

end

请先登录，再进行评论。

Answer 2

matlabUser 2011-10-12

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/17952-parallelizing-optimization-tasks-for-functions-evaluated-by-external-devices#answer_24405

在 MATLAB Online 中打开

OK, for those who are interested I am putting a working script below (Minimization of the two functions in parallel using spmd construct). I am using a simple quadratic function as an example of "external hardware function".

function [y1,y2] = process(x1, x2)
    y1 = (x1-15).^2;
    y2 = (x2-27).^2;
    pause(.1);    % putting a pause here to imitate hardware delay
end

First, you have to create an "argument extractor function" and put it in the same directory, out of the main script (spmd does not like handles to nested functions):

    function y = argExtractor(x)
        mediator = 3;
        % extracting arg x which labs 1&2 send to the mediator         
        labSend(x,mediator,1);  % data=x, destination=mediator, tag = 1
        y = labReceive(mediator);
    end

Then, in the main script put this:

    scheduler = findResource;
    scheduler.ClusterSize = 3;
    matlabpool(3)
    spmd
        % Specify the mediator lab
        mediator = 3;    
      if labindex == mediator
          % Initialize data
          data1 = 0; data2 = 0;
          while (~isnan(data1)) || (~isnan(data2))
              % Check if data is available from lab 2 and receive it
              if labProbe(1,1) && labProbe(2,1)           
                  data1 = labReceive(1,1);
                  data2 = labReceive(2,1);
                  [result1 result2] = process(data1,data2);
                  disp('HW functions 1&2 evaluated');
                  labSend(result1,1);
                  labSend(result2,2);
              elseif labProbe(1,1) && (labProbe(2,2) || isnan(data2))
                  data1 = labReceive(1,1);
                  data2 = NaN;  % 
                  [result1 result2] = process(data1,data2);
                  disp('HW function 1 evaluated');
                  labSend(result1,1);
              elseif (labProbe(1,2) || isnan(data1)) && labProbe(2,1)  
                  data1 = NaN;  % 
                  data2 = labReceive(2,1);
                  [result1 result2] = process(data1,data2);
                  disp('HW function 2 evaluated');
                  labSend(result2,2);             
              elseif (labProbe(1,2) || isnan(data1)) ... 
                     && (labProbe(2,2) ||  isnan(data2))
                  data1 = labReceive(1,2);  %NaN;
                  data2 = labReceive(2,2);  %NaN;
                  disp('Exiting because both optimizers finished');
              end
          end         
      end 
      if labindex < 3 
         % perform optimization: 
          [Xout,fval,exitflag] = fminunc(@argExtractor,5);
          labSend(NaN,mediator,2);
      end 
  end
% Print the results
X1out = Xout{1}
X2out = Xout{2}
exitFlag1 = exitflag{1}
exitFlag2 = exitflag{2}
matlabpool close

2 个评论
显示无隐藏无

Walter Roberson 2011-10-12

Trivial change:

if labindex < 3

should be

if labindex < mediator

matlabUser 2011-10-13

Hi, I have got another problem: the handle of the "external hardware function" is not passed to the "mediator" lab properly. My "external function" is an anonymous function defined outside spmd, and in its definition I use a handle to some application:

ExtFunctionHandle = @(X1,X2) ExternalFunction(X1,X2, ..., oApp, ...)

Here oApp is an application object which I opened on the client using ActiveX server:

oApp = actxserver('hardware_application');

(I control my external device via a special application by making assignments like oApp.register_in = Value, to send arguments, and result = oApp.register_out to read back the function values.)

I called this function handle right before spmd, and saw no problems. But when my mediator lab tries to call it, error message appears: "Trying to reference a field of an un-structured object". So, my lab worker recognizes the anonymous function handle, but the application object is not passed form the client to the lab... I tried to open the application object and create an anonymous function handle within lab worker to make are app object is passed correctly, but I am getting errors saying I can't create anonymous function within spmd body. So here is a dilemma... Any ideas? -Thanks!

请先登录，再进行评论。

parallelizing optimization tasks for functions evaluated by external devices

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

3 个评论
显示 1更早的评论隐藏 1更早的评论

更多回答（1 个）

2 个评论
显示无隐藏无

另请参阅

类别

标签

Community Treasure Hunt

parallelizing optimization tasks for functions evaluated by external devices

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

采纳的回答

3 个评论 显示 1更早的评论隐藏 1更早的评论

更多回答（1 个）

2 个评论 显示 无隐藏 无

另请参阅

类别

标签

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

3 个评论
显示 1更早的评论隐藏 1更早的评论

2 个评论
显示无隐藏无