memmapfile in parfor loops
显示 更早的评论
Hello, I'm trying to use mem mapped files for input and output of data from the body of a parloop.
If I understand correctly, this seems impossible, as the way memmapfile data is accessed (struct) violates the 'first indexing' rule for sliced variables in parallel loops.
As the memmapfile can be accessed by concurrent processes (as in the demo), is there a workaround to use it in parfor loops?
Thanks. L
回答（1 个）
Sarah Wait Zaranek
2011-11-11
You should be able to use memmapfile in a parfor loop. The trick is to pass m into a function within the parfor loop. You can get around the restriction that way. See simple example below:
function parMemMap
% randData = gallery('uniformdata', [100, 1], 0, 'double');
% fid = fopen('myfile.dat');
% fwrite(fid, randData, 'double');
% fclose(fid);
m = memmapfile('myfile.dat','Format', 'double','Writable',true);
parfor ii=1:length(m.Data)
output(ii,1) = myTestFunc(m,ii);
end
data = m.Data(:);
disp(output)
isequal(output,data)
end
function test=myTestFunc(m,ii)
m.Data(ii) = m.Data(ii)*rand;
test = m.Data(ii);
end
3 个评论
Adam Wyatt
2015-5-3
Is this safe to do?
I've had issues with corrupting files when accessing inside par-for loops. I can imagine however that because memmapfiles utilize native OS virtual memory routines, it should be multithread safe.
Keith
2024-2-10
11/11/11! Suppose there's a need to share data not only inside a parfor but between ML instances? Will a memmapfile, specifiying the desired disk file, in one thread facilitate reading that disk file as written by another(cooperating) thread?
Walter Roberson
2024-2-10
Will a memmapfile, specifiying the desired disk file, in one thread facilitate reading that disk file as written by another(cooperating) thread?
No.
However, a memmapfile in one thread could obtain data that you exchange with the client using parallel.pool.DataQueue and your client could then make that data available to another thread using a different set of parallel data queues. It is a somewhat ackward process.
类别
在 帮助中心 和 File Exchange 中查找有关 Large Files and Big Data 的更多信息
2024-2-10
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
