Coordinate workers in parallel generic algorithm optimization

3 次查看(过去 30 天)
Hi,
I am doing an optimization using the generic algorithm. It is a very long optimization and I am using the parallel calculation.
The cost function is long above does multiple things. To simplify, let's say that does the following: 1.-calculates a data set 2.-gets a sample 3.-calculates the cost
During step one (data set calculation) it uses a lot of ram, and ram is the bottleneck.
During step three (cost calculation), this is no longer the case, given that I am using a sample.
I can not use the sample to start with, unfortunately (before taking a sample I need to calculate running averages that depend on the last n points).
I have 6 cores available. During step 1 I can use at most 3 (to avoid running out of memory). During step 3 I can use all 6.
Is there a way to coordinate the 6 workers during execution?
I have tried using final calculates, keeping a count of the number of workers in each step and using the pause function for a worker of the number of workers in the next step is to high. For some reason this does not work, perhaps the global variables are only updated once the execution of the cost function terminates, but I need the global variables to be updated several times during the execution of the cost function.
Any ideas on how to solve?
  3 个评论
Grant
Grant 2016-2-25
Have you considered using parfeval? It allows asynch calculations in parallel and you should only assign one worker per function call.
The data you calculate will be 100% separated from each other so what you could do is have it run a fixed number of iterations then gather the data, or you could use synchronized function calls so that it will be able to gather the data at the same time if its really important.
I will say that parfeval() allows you to gather data as it becomes available.
Joe
Joe 2016-3-6
parfeval is a great learning from today, thanks for letting me know! I don't think I can use it for this particular case. I should program the generic algorithm myself, and I really want to avoid that.
What I ended up doing was doing the calculation with 3 cores...

请先登录,再进行评论。

回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Genetic Algorithm 的更多信息

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by