Why clearing memmapfile is taking too long?

1 次查看（过去 30 天）
yash
yash 2024-4-15
回答： Hassaan 2024-4-15
Hi team,
I am trying to use fileDataStore with custom read function. In this custom read function I am using memmapfile to read data from file. In this function I am clearing memmapfile object. It is taking approximately 3 seconds to get cleared. So, my questions are:
1) Why it is taking too long to clear memmapfile?
2) Is there any other way to clear memmapfile?
Following is attached snippets for the code.
[filename,filepath] = uigetfile('*.*', 'Select a file');
readTime = ReadDataStoreMapTime(fullfile(filepath, filename));
function time = ReadDataStoreMapTime(filename)
tic;
chunksize = 2.5*1024*1024*1024;
ds = fileDatastore(filename, 'ReadFcn', @readBinaryTRCFile, ...
'FileExtensions', '.trc', 'ReadMode', 'bytes');
ds.BlockSize = chunksize;
while hasdata(ds)
dataChunk = read(ds);
end
time = toc;
end
function data = readBinaryTRCFile(filename, offset, size)
tic;
size = floor(size / 8);
file = memmapfile(filename, ...
'Offset', offset, ...
'Writable', true, ...
'Format', 'double', ...
'Repeat', size);
data = file.Data;
clear file;
disp(toc);
end

请先登录，再回答此问题。

回答（1 个）

Hassaan
Hassaan 2024-4-15
function time = ReadDataStoreMapTime(filename)
tic;
chunksize = 2.5*1024*1024*1024;
ds = fileDatastore(filename, 'ReadFcn', @readBinaryTRCFile, ...
'FileExtensions', '.trc', 'ReadMode', 'bytes');
ds.BlockSize = chunksize;
while hasdata(ds)
dataChunk = read(ds);
end
time = toc;
end
function data = readBinaryTRCFile(filename, offset, size)
tic;
size = floor(size / 8);
file = memmapfile(filename, ...
'Offset', offset, ...
'Writable', true, ...
'Format', 'double', ...
'Repeat', size);
data = file.Data;
disp(toc); % The memmapfile object 'file' will be cleared automatically when it goes out of scope
end
By relying on MATLAB's automatic cleanup when file goes out of scope, this might slightly improve the performance or at least the predictability of memory management. If these steps don't help, consider profiling the code with MATLAB's profiler to identify any other bottlenecks or issues.

类别

在 Help Center 和 File Exchange 中查找有关 Parallel Computing 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by