- Is it in a script or function outside all of the classes you're trying to use?
- Is it in a method of one or more of those classes (and if so what kind of method: ordinary, constructor, static, something else?)
Parfor with user made classes
9 次查看(过去 30 天)
显示 更早的评论
Hello.
I am using a user made toolbox in my current project and with that user made classes. Now I have two for-loops which by its defined purpose will always have to deal with larger data amounts, where the first one allocates and the second one computes. As an example, the first loop looks just like
for i = 1 : els
switch etype(strct(i))
case 'sref'
% Do something
case 'aref'
% Do something
case {'boundary', 'path'}
% Do something
otherwise
% Do something drastic
end
end
etype is also part of the user made library and returns some of the names in the cases. There have been no issues with the library. The code works within for-loops.
Now considering the data amount will always be large and there has to be pre-allocation in some way, I wanted to try to decrease runtime by using parfor, since the do somethings should be separable. But if I exchange the for with parfor, I get the following warnings:
Warning: Element(s) of class 'gds_element' do not match the current constructor definition. The element(s) have been converted to structures.
> In parallel.internal.pool.deserialize (line 33)
In remoteParallelFunction (line 66)
Warning: Element(s) of class 'gds_element' do not match the current constructor definition. The element(s) have been converted to structures.
Warning: Element(s) of class 'gds_element' do not match the current constructor definition. The element(s) have been converted to structures.
Analyzing and transferring files to the workers ...done.
Warning: Element(s) of class 'gds_element' do not match the current constructor definition. The element(s) have been converted to structures.
> In parallel.internal.pool.deserialize (line 33)
In remoteParallelFunction (line 66)
Warning: Element(s) of class 'gds_element' do not match the current constructor definition. The element(s) have been converted to structures.
which culminates in the following error
Error using downstream
The source code [url] for the parfor-loop that is trying to execute on the worker could not be found.
Caused by:
Undefined function 'etype' for input arguments of type 'cell'.
Error using remoteParallelFunction
Worker unable to find file.
Undefined function 'etype' for input arguments of type 'cell'.
Is it the case that Matlab just can't handle any user defined classes in parallel loops? Is that a general thing? Or did I make a mistake? Is it because of the Toolbox?
I do understand that etype can't deal with cells - an annoyance of the toolbox - but I'm not passing a cell but specifically a gds_element. Even a workaround a la
elem = strct(i);
if iscell(elem); elem = elem{1}; end
switch etype(elem)
didn't help.
2 个评论
Steven Lord
2023-3-1
Where is the code you wrote in your first code block located?
Are you creating the objects before the parfor loop and using them inside the loop or does your code inside the loop attempt to create the objects?
回答(1 个)
Raghav
2023-5-4
Hi,
Based on the question, it can be understand that you are facing compatibility issues of user made classes with Parallel computing toolbox.
The warnings and error message you are seeing suggest that there might be compatibility issues with the user-defined classes and the parallel computing toolbox in MATLAB. When using parfor in MATLAB, the data that is being passed between the worker processes needs to be serialized, sent across to the worker processes, and then deserialized. This process can result in issues with custom classes that are not serializable, or whose definitions are not available on the worker processes.
It's possible that the gds_element class is not serializable or not defined on the worker processes, which is causing the errors you're seeing. You may need to modify the implementation of your gds_element class to ensure that it is compatible with the parallel computing toolbox.
One thing you could try is to modify your code to pass only the necessary data that is needed in the loop to the worker processes, rather than passing the entire gds_element object. For example, you could extract the relevant data from the gds_element object and create a new struct that only contains the necessary data. This could help to reduce the size of the data being passed between the worker processes and potentially improve performance.
Alternatively, you could try using the spmd construct instead of parfor, which is designed for parallel computing with MATLAB. spmd allows you to execute code on multiple MATLAB workers and explicitly control the data that is passed between the workers. This could be a better option if you are dealing with custom classes or data structures that are not compatible with parfor.
The official documentation for spmd is mentioned below:
Hope it helps,
Raghav Bansal
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Parallel Computing Fundamentals 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!