`parfor` doesn't seem to work with Python lists

Question

Hagai Hargil 2016-10-10

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/306676-parfor-doesn-t-seem-to-work-with-python-lists

评论： Robert Snoeberger 2016-10-12

采纳的回答： Robert Snoeberger

在 MATLAB Online 中打开

Hellp everyone,

I'm trying to use a parfor loop that includes a Python list as an input argument to a class array:

 allEvents(1:size(hexData, 2), 1) = Event(); 
 parfor n = 1:size(hexData, 2)
      allEvents(n) = createSingleEvent(hexData{n}, dataFormat, range, hexToBinMap);
 end

where hexData{n} is the mentioned Python list, and allEvents is a preallocated array of class Event() (value class, not handle).

Unfortunately, no matter how I turn it, the parfor loop won't run. Needless to say, a regular for loop compiles properly.

The error I receive is a "Python error: Index exceeds list dimensions". When I manually check possibly-problematic indices I recieve no such error.

By adding a "char(hexData{n})" inside the parfor I receive "Index exceeds matrix dimensions" in the first occasion inside the function where I need to use the hexData{n} value, even if I see that the index definitely exists in that variable:

startOfString = hexDataAsChar(1); % This errors out

Only when I erase all traces that hexData was ever a Python list does MATLAB kindly allows me to parfor this loop.

Any help will be deeply appreciated. Thanks!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Robert Snoeberger 2016-10-11

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/306676-parfor-doesn-t-seem-to-work-with-python-lists#answer_238539

If you are using a parpool, then parfor needs to save and load the Python list. Saving (serializing) Python objects into a MAT-file is listed as a limitation .

2 个评论
显示无隐藏无

Hagai Hargil 2016-10-12

Dear Robert, Thank you for your quick reply. I wasn't aware that this 'save and load' process occurs during the parfor loop. Is there a workaround around this limitation, which can enable me to parallelize processing of Python objects?

Robert Snoeberger 2016-10-12

Unfortunately, I'm not aware of a workaround.

The only idea that I can think of is to wrap the Python object in your own MATLAB class with customized save and load . You might be able to use pickle to do the serialization. I haven't tried it so I don't know if the idea is viable.

请先登录，再进行评论。