How to import large data file in function in matlab?

2018 7 26
1 个回答
更新时间：2025 2 4
6 次查看（30 天）

0 个投票

I have a satellite data values stored in a .dat file of almost 900 MB. The data is in the form of int8. I want to read as much data as possible from the start and then pass it in a function argument and then save the new signal in a separate file. I have used importdata but it cant work and unable to read large data file. Is there any other way which I can use to read data in chunks and put them together afterwards? Anyone please help me out. Its very important that I read this data.
The specifications of software, platform & PC are:
Matlab R2015a. Windows 7 64-bit, Core i5 1st Gen., 4 GB RAM, 250 of hard Disk Space

8 个评论
显示 6更早的评论 隐藏 6更早的评论

Bob Thompson
Bob Thompson 2018-7-26
Is it possible to break apart the file manually, and read the different sections?
Sheikh Alsheikh
Sheikh Alsheikh 2018-7-26
actually I have no knowledge about number of rows and column of data or synchronization, i am unable to break or even process my file whenever i give any command it turn into long lasting processing so i have to terminate
Bob Thompson
Bob Thompson 2018-7-26
编辑：Bob Thompson 2018-7-26
Does Matlab give you a memory error? Or does it just take forever to load? 900MB is a pretty big file, so it will take Matlab a while to process it (probably several minutes).
An alternative option would be to open the file in a text editor (Textpad, Wordpad, etc.), and try to break the file apart that way.
Sheikh Alsheikh
Sheikh Alsheikh 2018-7-27
Dear Bob When i used the Matlab Build in import data button an upload file in it , it show me whole file in three different variable in form just like excel format
Sheikh Alsheikh
Sheikh Alsheikh 2018-7-27
Thanx , dpd but i have to upload y file in a function argument
Guillaume
Guillaume 2018-7-27
The data is in the form of int8
What does that mean? Is the file a text file (as Bob assumed) or a binary file. Loading a 900MB binary file shouldn't take long.
dpb
dpb 2018-7-27
"have to upload y file in a function argument"
And what does that mean?
You have a large file; TMW supplies several tools to deal with those but to give any specific help we need at least a klew as to what it is that is the real problem, also specifically.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Karan Singh
Karan Singh 2025-2-4

0 个投票

Hi @Sheikh Alsheikh,
I think I have the perfect solution for you! You can check out this MathWorks documentation ( https://in.mathworks.com/help/matlab/large-files-and-big-data.html ), as it provides relevant details. I believe "Datastore" would be the best approach for your needs. https://in.mathworks.com/help/matlab/datastore.html
Karan

类别

在 帮助中心 和 File Exchange 中查找有关 Large Files and Big Data 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by