How can I load a mat file faster

Question

Luis Garcia 2018-12-21

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/436854-how-can-i-load-a-mat-file-faster

回答： Jan 2018-12-22

I have a very large mat file that is about 8 GB,it contains a couple of structure with data. And to load it it takes about 10 minutes with an SSD. Is there any other way to load it faster. I use the v.73 flag because the file was more than 2GB.

Thank you.

2 个评论
显示无隐藏无

Walter Roberson 2018-12-21

How long does it take to fileread() the .mat file? The timing for that would give an indication of how much time is being spent on I/O compared to how much time is being spent decoding what is being read.

If the amount of time spent just reading from SSD is more than the number of structures times faster than decoding in load(), then you might be able to take advantage of using parfor to load() the different structures in different processes, distributing the decoding work.

per isakson 2018-12-21

Are these structures complicated and have zillions of fields?

Do you use '-nocompression'?

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Jan 2018-12-22

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/436854-how-can-i-load-a-mat-file-faster#answer_353584

You can load the file faster by using faster hardware: SSD, RAM and CPU matter. Maybe your RAM is exhausted and the computer uses theSSD as virtual memory. If you have stored the file with compression, the decompressed data might be much larger than 8 GB. Then installing more RAM will help. Or the MAT file contains user-defined objects, which need a time-consuming initialization - we cannot guess such details.