How to load a mat. file containing one big matrix

13 次查看（过去 30 天）
Klara Stribrna
Klara Stribrna 2023-3-9
评论： Walter Roberson 2023-3-10
Hello, I am working with 23.8 GB mat. file so I can't load it with the load function. I've tried to load it with matfile function but I have found out that in only contains one variable, which is a 1x1 struct and is again too big to be loaded. I've tried to read one element of the matrix (it's 4-dimensional) and it didn't work, still the same error ("Requested 994x961x3346x1 (23.8GB) array exceeds maximum array size preference (15.8GB). This might cause MATLAB to become unresponsive.") The mat. file was generated from an external program and I think the problem might be that it was not saved with the -v7.3 flag so it doesn't support partial loading. Is there a way to resave this file with the -v7.3 flag without having to load it all first? Or do you have any other suggestions how to acces this data?
  6 个评论
显示 4更早的评论
Bruno Luong
Bruno Luong 2023-3-10
Thanks. Is there any difference (performance)
  • The array size limit is unchecked?
  • Checked with slider at 100% (default)?
Stephen23
Stephen23 2023-3-10
"Is there any difference (performance)"
As far as I can recall, if the size limit is unchecked then performance is unaffected (at least, nothing humanly noticeable) until an array larger than the installed memory is requested, then the virtual memory is used and it is slower.
But still quite useable if using an SSD.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Jan
Jan 2023-3-9
编辑：Jan 2023-3-10
If the variable in the MAT file has more than 2GB, it must have the v7.3 format.
You try to extract a 994x961x3346 subarray, which needs 25.57 GB for the type double (The difference to 23.8 is ([EDITED], "most likely" removed) a mixing of GB and GiB). But you explain, that the complete file has 23.8 GB also?
If the wanted submatrix does not match into your RAM, this is not a problem of the MAT file, but of an insufficient amount of installed RAM. So install more RAM or import parts of the submatrix only.
  11 个评论
显示 9更早的评论
Walter Roberson
Walter Roberson 2023-3-10
If you are talking about RAM, for historical reasons it is very common to use 1024 as the multiplier. For example 8 GB RAM is typically 8 * 1024^3 -- at least when you are talking in terms of how much RAM is installed or what the limits are for linking or program size. But if you are looking at how many kilobytes a program is using in the process monitor, there is a tendancy to slip into 1000 as the prefix.
But when you are talking about hard disks especially, 1000 has used as the multiplier for quite a number of years now; possibly not originally but the marketing department won the battle with hard disks a long time ago.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Data Type Conversion 的更多信息

标签

产品


版本

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by