memmapfile in parfor loops

Question

Lorenzo 2011-11-10

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/20848-memmapfile-in-parfor-loops

评论： Walter Roberson 2024-2-10

Hello, I'm trying to use mem mapped files for input and output of data from the body of a parloop.

If I understand correctly, this seems impossible, as the way memmapfile data is accessed (struct) violates the 'first indexing' rule for sliced variables in parallel loops.

As the memmapfile can be accessed by concurrent processes (as in the demo), is there a workaround to use it in parfor loops?

Thanks. L

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Sarah Wait Zaranek 2011-11-11

3
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/20848-memmapfile-in-parfor-loops#answer_27721

在 MATLAB Online 中打开

You should be able to use memmapfile in a parfor loop. The trick is to pass m into a function within the parfor loop. You can get around the restriction that way. See simple example below:

function parMemMap
% randData = gallery('uniformdata', [100, 1], 0, 'double');
% fid = fopen('myfile.dat');
% fwrite(fid, randData, 'double');
% fclose(fid);
m = memmapfile('myfile.dat','Format', 'double','Writable',true);
parfor ii=1:length(m.Data)
    output(ii,1) = myTestFunc(m,ii);
end
data = m.Data(:);
disp(output)
isequal(output,data)
end
function test=myTestFunc(m,ii)
m.Data(ii) = m.Data(ii)*rand;
test = m.Data(ii);
end

3 个评论
显示 1更早的评论隐藏 1更早的评论

Keith 2024-2-10

11/11/11! Suppose there's a need to share data not only inside a parfor but between ML instances? Will a memmapfile, specifiying the desired disk file, in one thread facilitate reading that disk file as written by another(cooperating) thread?

Walter Roberson 2024-2-10

Will a memmapfile, specifiying the desired disk file, in one thread facilitate reading that disk file as written by another(cooperating) thread?

No.

However, a memmapfile in one thread could obtain data that you exchange with the client using parallel.pool.DataQueue and your client could then make that data available to another thread using a different set of parallel data queues. It is a somewhat ackward process.

请先登录，再进行评论。