How to import large textfile

Question

Rajab Omar 2014-6-9

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/133420-how-to-import-large-textfile

编辑： Cedric 2014-6-12

test.txt

Hi EVERYONE I'm new to matlab, I've been trying to import large text file .asc to matlab. is there anyone can help. the file contains 59615 (24*24)matrix. I've attached small part of the file. When I import the whole file I get an error while if I import small part it works fine!! I don't think it's memory problem, can anyone help please???

4 个评论
显示 2更早的评论隐藏 2更早的评论

Cedric 2014-6-9

编辑：Cedric 2014-6-9

在 MATLAB Online 中打开

It is not too large, but not small either. Assuming 10 chars per element (including separators, etc), it is more than 300MB

 >> 59615*24*24 * 10 / 1e6
 ans =
  343.3824

Could you provide a ~5MB sample?

Rajab Omar 2014-6-9

在 MATLAB Online 中打开

test.txt

here's the error I got: "import operation failed. the most likely reason is that there are unimortable cells in the selection. try to adding arule in the toolstrip to convert unimortable cells into numbers."

I've a;ready excluded the raws with unimortable cells.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Cedric 2014-6-9

2
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/133420-how-to-import-large-textfile#answer_140132

编辑：Cedric 2014-6-9

在 MATLAB Online 中打开

Here is one way to proceed

 fId = fopen( 'omar_1.txt', 'r' ) ;
 buf = fscanf( fId, '%f', Inf ) ;
 fclose( fId ) ;
 bSize   = 24*24 + 1 ;
 nBlocks = length( buf ) / bSize ;
 data    = cell( nBlocks, 1 ) ;
 for bId = 1 : nBlocks
    bBnds     = ((bId-1)*bSize+2) : (bId*bSize) ;
    data{bId} = reshape( buf(bBnds), 24, 24 ).' ;
 end

At the end, you have your matrices in the cell array data:

 >> data
 data = 
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]
    [24x24 double]

so data{1} is the first matrix. There are other approaches, but they a probably less efficient if your file is really large.

4 个评论
显示 2更早的评论隐藏 2更早的评论

Rajab Omar 2014-6-12

编辑：per isakson 2014-6-12

在 MATLAB Online 中打开

Thanks a lot Cedric Wannaz I've different question now, I'm not sure if it's OK to ask it within this question. I've got time series data and need to get the power spectral density (PSD), and the probability density function (PDF). I wrote code for PSD, then I realised that there's a function called (pwelch). However the results were not identical. I shall be grateful if you can suggest the best option to estimate both PSD and PDF for time series data. HERE'S WHAT I'VE DONE:

    %clf;clc;clear;
    VFF_1=VFF.*hanning(length(VFF));%%windowing
    N= length(VFF_1);
    dt=0.001;
    fs=1/dt
    tmax=(N-1)*dt;
    t=0:dt:tmax;
    tn=N/fs%%Total sample length
    fmin=1/tn
    fmax=0.5*fs;%%nyquist
    f=(1:N/2)/(N/2)*fmax;%define frequency
    f=f';
    length(f)
    y=fft(VFF_1);
    y(1)=[];
    py=abs(y(1:N/2)).^2;%%Define power spectrum
    length(py)
    figure
    plot(f,py)%%plot power spectrum against Frequency
    xlabel('Frequency (HZ)');title('Power Spectrum');ylabel('power');

REGARDS

Cedric 2014-6-12

编辑：Cedric 2014-6-12

Hi, as it is a different topic, you should post a new question. You'll also get answers from people who are more proficient than I on this particular matter!

Regards,

Cedric

请先登录，再进行评论。