Parallel Computing Toolbox - Java object transfer from client to workers

2015 7 21
1 个回答
回答已采纳
更新时间：2015 7 27
8 次查看（30 天）

0 个投票

My problem regards the usage of the Parallel Computing Toolbox functions, and in particular the variable transfer from client to workers when the "variable" is a Java object. I have experieced that, without multithreading, passing a Java object from a main script to a function is not a problem, while using (for instance) parfeval to implement multithreading, the object is not passed correctly resulting in the error "Attempt to reference field of non-structure array", because the object structure is lost. Something similar happens to me also trying to implement multithreading exploiting a parfor cycle, therefore I suspect it may be a generalized issue with the usage of Matlab multithreading tools. May I kindly have any clarification with this regards?

请先登录，再回答此问题。

 采纳的回答

Edric Ellis
Edric Ellis 2015-7-21

2 个投票

When you pass any MATLAB variables (including those that refer to Java objects) into parfor, parfeval, etc., then the contents of those variables need to be transmitted to a separate MATLAB process. The effect is the same as if you saved the variables to a .mat file and then loaded them. Not all MATLAB variables can be saved and loaded successfully - and those same variables will not function correctly with parfor, parfeval etc.
I recommend that you should construct such objects on the workers (i.e. inside parfor, parfeval, ...).
You might wish to use the Worker Object Wrapper to avoid constructing the same object multiple times on the workers.

4 个评论
显示 2更早的评论 隐藏 2更早的评论

Stefano C
Stefano C 2015-7-22
编辑：Stefano C 2015-7-22
Many thanks for your answer Edric, unfortunately WorkerObjWrapper does not seem to work for me, in fact having this situation:
main.m
[...]
wrapperData = WorkerObjWrapper(@generateDataset, fName, numFolds);
for fold = 1 : numFolds
% call Parallel Function Evaluation
CV(fold) = parfeval(p, @CVloop, 2, wrapperData.Value, numFolds, fold);
end
for fold = 1 : numFolds
% fetchNext blocks until next results are available.
87: [completedCV,value] = fetchNext(CV);
fprintf('Got result #%d\n', completedCV);
end
[...]
CVloop.m
[...]
23: trainSet = randData.trainCV(numFolds, fold - 1);
[...]
I still get the error:
Error using parallel.FevalFuture/fetchNext (line 243)
The function evaluation completed with an error.
Error in main (line 87)
[completedCV,value] = fetchNext(CV);
Caused by:
Error using CVloop (line 23)
Attempt to reference field of non-structure array.
Depite of that, using dedicated functions to create the Java objects for each worker, as you suggested, everything seems to work fine, even if not very efficiently (but acceptably).
Thanks again
Edric Ellis
Edric Ellis 2015-7-23
编辑：Edric Ellis 2015-7-23
You should call
CV(fold) = parfeval(p, @CVloop, 2, wrapperData, numFolds, fold);
to send the whole wrapper to the worker, and then only on the worker pick out the .Value field. (And just to be sure - you should be creating the WorkerObjWrapper and calling parfeval at the client).
Stefano C
Stefano C 2015-7-26
编辑：Stefano C 2015-7-27
Thanks a lot Edric, sending the whole wrapper I now get a different error:
Error using WorkerObjWrapper
Too many input arguments.
Error in main (line 62)
wrapperData = WorkerObjWrapper(@generateDataset, WPATH, fName, numFolds);
It seems there is a sort of issue calling the wrapped function with multiple arguments, do you know why?
EDIT:
I have been able to solve this last question looking at the code of WorkerObjWrapper:
% W = WORKEROBJWRAPPER( FCN, ARGSCELL ) creates a WorkerObjWrapper by
% invoking FCN inside an SPMD block, supplying ARGSCELL as
% arguments. I.e., W.Value has the value obtained from running
% FCN(ARGSCELL{:}) on the workers.
Hence, for the posterity, in case the wrapped function has multiple inputs, WorkerObjWrapper has to be called in this way:
FcnArgs = {arg1, arg2, arg3, ...};
wrapperData = WorkerObjWrapper(@Fcn, FcnArgs);
Stefano C
Stefano C 2015-7-27
编辑：Stefano C 2015-7-27
Excuse me Edric, despite having (I think) correctly instantiated the objects wrappers on the client and picked out the .Value on the workers, still WorkerObjWrapper does not seem to work, I now get the error:
Error in main (line 90)
[completedFold,value] = fetchNext(CV);
Caused by:
Error using WorkerObjWrapper/get.Value (line 109)
Assertion failed.
kindly, would you know what it may depend upon?

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Parallel for-Loops (parfor) 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by