Caught std::exception Exception message is: bad allocation

10 次查看（过去 30 天）
Bernard
Bernard 2012-2-13
Hi to all!
I am trying to run the code below as suggested by Andrei in this forum to get a data set converted into a 633 by 150005 Matlab matrix for further analysis. The data file itself consists of 633 lines with at most 150005 characters per line plus 1897 headerlines on top of it.
fid = fopen('datafile.txt');
t = textscan(fid,'%150005c','HeaderLines',1897,'BufSize',160000);
fclose(fid);
out = reshape(regexprep(t{1}(:)',{'o',' ','"',','},{'1' '0' 'b' 'e'}),size(t{1}))-'0';
Experiments with 2 lines of 150005 characters yielded a correct 2 by 150005 Matlab variable, at least by changing the BufSize to 160000. However, when I try to run the scripts above using my "real data", it gave the following message and no result at all.
Caught std::exception Exception message is: bad allocation
By typing in memory at the prompt, I get what follows. This may be helpful in resolving the issue.
Maximum possible array: 223 MB (2.340e+008 bytes) *
Memory available for all arrays: 775 MB (8.129e+008 bytes) **
Memory used by MATLAB: 1028 MB (1.078e+009 bytes)
Physical Memory (RAM): 2031 MB (2.129e+009 bytes)
* Limited by contiguous virtual address space available.
** Limited by virtual address space available.
I attempted pre-allocating by creating zeros(633,150005) but I get "Out of memory. Type HELP MEMORY for your options." Please help.
Thanks,
Bernard
  2 个评论
Walter Roberson
Walter Roberson 2012-2-13
http://www.mathworks.com/matlabcentral/answers/28742-converting-a-text-file-to-a-matlab-matrix-variable

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Walter Roberson
Walter Roberson 2012-2-13
Read a line at a time and covert it and store it. The arrangement you have now requires two copies in memory at the same time (original and translated), and that would go through your memory limit even if you were not using 8 bytes per character in the translated output.
As you only have 4 distinct values (it appears from the above), you should store the results as uint8 to save memory.
  1 个评论
Bernard
Bernard 2012-2-14
Walter, thank you for the advice... Does that mean that I have to back to my old scripts as shown below?
for n=1:50,000;
fid=open('datamap.txt')
for k=1:633;
tline=fgets(fid);
if isletter(tline(n))==1;
Matlab(k,n)=1;
else Matlab(k,n)=0;
end
end
fclose all
end
I find this procedure very slow.... but I'l give it a try though... Do you have any suggestions?
Thanks,
Bernard

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Logical 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by