read file variables methods
显示 更早的评论
I got a large data matrix (3Gb) of signal data which I saved to a .mat file. Since the data size is large I figured out it will be wiser to access the file and read each time a block of data, process it and move to the next block instead of loading the complete file. I'm wondering should I use the matfile method, fopen and fread or should I use a datastore and read command. What is faster from the processing point of view and easier to work with? Or any other considerations? Thanks
3 个评论
per isakson
2016-11-11
"data set"   is that one numerical array?
"fairly large data set"   how large compared to the physical memory?
Talmon Alexandri
2016-11-12
per isakson
2016-11-14
Given
- "large data matrix (3Gb) of signal data which I saved to a .mat file"
- "Total RAM size - 3G[B]"
- "[one] plain numerical array 10 million lines by 48 columns"
I assume the mat-file is version 7.3, because that's needed to store an array of that size.
I would say this a a perfect case for matfile, Access and change variables directly in MAT-files, without loading into memory.
It's the easiest to work with and I don't think there is any other way that is significantly faster.
回答（1 个）
Walter Roberson
2016-11-11
0 个投票
类别
在 帮助中心 和 File Exchange 中查找有关 Large Files and Big Data 的更多信息
2016-11-11
2016-11-14
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
