Speeding up a load file workflow

3 次查看(过去 30 天)
I'm working with a program that outputs simulation to results to many .mat files. To analyze these results, I need to load each of these mat files. Currently, I use the following lines to do this.
for i = 1:numFiles
cases{i} = load(files(i).name);
end
after preallocating the cases cell array, of course. The problem is, repeatedly using the load function like this can drastically increase the time I need to analyze my results, depending on how many mat files this program outputs.
My question, then, is this: Is there any way to load multiple files at once?
Thanks!

采纳的回答

Swastik Sarkar
Swastik Sarkar 2024-11-22
Hi @Paul,
I know of 2 options to load the MAT-files faster, both of which will require the Parallel Computing Toolbox.
One approach is to utilize the parfor loop to load MAT files in parallel on separate workers:
parfor i = 1:numFiles
cases{i} = load(files(i).name);
end
Another approach is to use the parfeval function to create futures and wait for them asynchronously:
for k = 1:numFiles
futures(k) = parfeval(@load, 1, files(k).name);
end
for k = 1:numFiles
[idx, loadedData] = fetchNext(futures);
cases{idx} = loadedData;
end
Hope this helps load MAT-files faster.
  2 个评论
Paul
Paul 2024-11-22
I accepted this because you have answered my question. If I don't want to use the PCT, do you know of any ways to do that?
Walter Roberson
Walter Roberson 2024-11-22
load() is compatible with using backgroundPool and parfeval
Other than Parallel Computing Toolbox, and Background Pools, there is no way to load multiple files simultaneously.

请先登录,再进行评论。

更多回答(0 个)

类别

Help CenterFile Exchange 中查找有关 Programming Utilities 的更多信息

产品


版本

R2024a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by