Fastest way to load data.

45 次查看(过去 30 天)
atharva aalok
atharva aalok 2023-8-16
编辑: dpb 2023-8-16
Hello,
I have an array of size ~320 Mb that I want to store in a file such that the loading afterwards is as fast as possible.
Currently simply doing:
save('filename.mat', 'var');
takes around 2.4 seconds.
What flags (if any) can I give the save() or load() functions that can achieve this?
Currently on saving as .mat file using the '-v6' flag (file size now 470Mb) has allowed me to get the load time down to 0.14 seconds!
Are there things I can leverage to reduce this further?
My SSDs read and write speeds are ~1.4 GB/s

回答(1 个)

dpb
dpb 2023-8-16
编辑:dpb 2023-8-16
save does compression by default now which takes time as you've discovered.
The absolute fastest albeit somewhat less convenient should be using fwrite, fread as far as pure i/o speed.
fid=fopen('filename.bin','w');
fwrite(fid,var)
fid=fclose(fid);
One test here was almost a tie, however, so likely not sufficiently faster to be worth the effort.
The way to speed it up if you can afford to lose some precision would be to only use single-precision instead of double...half as many bytes to read/write. It won't be a full 2X gain because memory access will be just a tad quicker for 8-byte rather than 4-byte increments probably, but should be observeable speedup. Of course, if you need to preserve full precision this isn't an option.

类别

Help CenterFile Exchange 中查找有关 Logical 的更多信息

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by