Handling memory issues by connecting to database and retrieving only required data
显示 更早的评论
I have an excel file as large as 20MB with roughly abt 1.5 million rows. I run out of memory while I perform any sort of operation on the data though I am able to read data completely from this huge file. As an alternative would it be a good idea to connect to a database and fetch only the data required for each of my operation (Although this could mean connecting to the database several number of times.) ?
采纳的回答
Walter Roberson
2011-3-7
It seems odd that you would run out of memory on basic operations on such a small file. You are only averaging about 12 characters per row, which would tend to suggest one double precision number per row, which would add up to only 12 megabytes of storage. That's relatively small.
Please check that you are not accidentally using "*" (matrix multiplication) instead of ".*" (element-by-element multiplication) in your routines.
Does even something as simple as
B = A + 0;
cause you to run out of memory? If not then just read the file once and access the sub-ranges you need. If it does run out of memory with something that simple, please use "whos" to figure out what is taking up so much memory. If you are working on the 32 bit Windows version, the command "memstats" can tell you about available memory.
更多回答（1 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Code Performance 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
