How to read large data file in Matlab?

27 次查看（过去 30 天）
Umair Nadeem
Umair Nadeem 2013-11-14
评论： Walter Roberson 2022-2-6
I have a GPS signal data values stored in a .dat file of 200 GB. The data is in the form of uint8. I want to read 16 GB of data from the start and downsample it by a factor of 10 before storing the new signal in a separate file. I have used fread but I cam unable to read large data file. Is there any other way which I can use to read data in chunks and put them together afterwards? Anyone please help me out. Its very important that I read this data.
The specifications of software, platform & PC are:
Matlab R2012a. Windows 8 64-bit, Core i5 3rd Gen., 4 GB RAM, 500 GB of hard Disk Space

请先登录，再回答此问题。

采纳的回答

Simon
Simon 2013-11-14
Hi!
Is it a binary file or a normal text file? For text files you may use "fopen" and "fgetl" to read line by line. If you know your row format you may use "fscanf" to read a given number of values. Repeated calls to "fscanf" will read your whole file.
Can you give some (!) example lines? It is easier to help.
  4 个评论
显示 2更早的评论
Sania Zahan
Sania Zahan 2022-2-6
Hi My data shape is (39412, 3, 300, 25, 2), its numpy file with just numbers. Therefore above method changes the shape. I need to maintain the last 4 indexes. could you please suggest how may I do that. Thanks
Walter Roberson
Walter Roberson 2022-2-6
A = fread(fid, [39412, 3, 300, 25, 2]);
However, you really need to test first, as it would be common to find the first two dimensions transposed between different programming languages. You might need to use
A = permute(fread(fid, [3, 39412, 300, 25, 2]), [2 1 3 4 5]);

请先登录，再进行评论。

更多回答（1 个）

Sheikh Alsheikh
Sheikh Alsheikh 2018-7-26
Same problem I had o face! I first break the data into chunks
  1 个评论
Hammad Intersoft
Hammad Intersoft 2021-5-26
how to open a large file by breaking it into chunks. i am facing a problem while opening a large file that has a size of 48*42*2414 and bytes 4866624. when i load this file in matla it says "cannot display summaries of variables with more than 524288 elements."

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Data Import and Analysis 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by