Textscan with blocks of data with headers to individual block

5 次查看（过去 30 天）
Raj Raj
Raj Raj 2014-10-25
回答： Gabriel Felix 2020-5-24
Hi All,
I am trying to extract to following data with textscan. But I am not able to implement it correctly. Each block consists of 11 data lines and 8 unnecessary lines to exclude.
Data file (testdata.dat) is :
Title
Data Header: Time: 100.16064 Sec 10/8/2012 3:32:17 PM
Data Acquisition: T
Station Name: default
Test File Name:
Axial Axial Axial
segments mm kN
0 -0.19470811 -0.0067140702
0 -0.19806515 -0.0033570339
0 -0.19806515 -0.0033570339
0 -0.19470811 -0.0067140702
0 -0.16785182 -0.0033570339
0 -0.16785182 -0.0067140702
0 -0.13763849 -0.0067140702
0 -0.12421035 -0.0067140702
0 -0.10742517 -0.0033570339
0 -0.11413924 -0.0033570339
0 -0.14099553 -0.0033570339
Data Header: Time: 100.16064 Sec 10/8/2012 3:32:17 PM
Data Acquisition: T
Station Name: default
Test File Name:
Axial Axial Axial
segments mm kN
0 -0.16785182 -0.0033570339
0 -0.20813625 -0.0067140702
0 -0.2316355 -0.0067140702
0 -0.25513476 -0.0067140702
0 -0.29206216 -0.0033570339
0 -0.35584584 -0.0067140702
0 -0.40620136 -0.0067140702
0 -0.47334209 -0.0067140702
0 -0.55055392 -0.0067140702
0 -0.61098057 -0.0067140702
0 -0.70162052 -0.010071106
My implementation is
fileID = fopen('testdata.dat');
formatSpec = '%d %f %f';
clear C
k = 0;
N = 11;%Block size
while ~feof(fileID)
k = k+1;
C(k,1:3) = textscan(fileID,formatSpec,N,'HeaderLines', 8);%'CommentStyle','##',
end
fclose(fileID)
Please correct my code. Thanks in advance.
  2 个评论
Raj Raj
Raj Raj 2014-10-25
p.s.
I tried the code given in Here However, I am getting error with my data file as
Index exceeds matrix dimensions.
at
ncol = numel( regexp( buf{1}, ' +', 'split' ) );
Appreciate your comments.
per isakson
per isakson 2014-10-27
编辑：per isakson 2014-10-27
That function read one trailing block of data after a file header, the last word of which was "END".

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

per isakson
per isakson 2014-10-27
编辑：per isakson 2014-10-27
Modified version of your code
fid = fopen('RajRaj.txt');
formatSpec = '%d%f%f';
clear C
k = 0;
N = 11; %Block size
while ~feof( fid )
k = k+1;
C( k, 1:3 ) = textscan( fid, formatSpec, N, 'HeaderLines', 8 );
end
fclose( fid );
check the output
>> whos C
Name Size Bytes Class Attributes
C 2x3 1112 cell
"Please correct my code" &nbsp I didn't try your code and isn't sure why it doesn't work. It's tricky to count header-lines. An ending new-line matters.
&nbsp
Here is another function that uses &nbsp fileread &nbsp and &nbsp regexp
>> g = read_blocks_of_numerical_data( 'RajRaj.txt', 150 );
>> g{:}
ans =
0 -0.1947 -0.0067
0 -0.1981 -0.0034
...
ans =
0 -0.1679 -0.0034
0 -0.2081 -0.0067
...
Where both the function and the sample text file are attached. The magical number &nbsp 150 &nbsp is the minimum number of bytes in a block of numerical data. The entire file must fit in memory together with the result.
&nbsp
And one more function
>> M = RajRaj()
M(:,:,1) =
0 -0.1947 -0.0067
0 -0.1981 -0.0034
...
M(:,:,2) =
0 -0.1679 -0.0034
0 -0.2081 -0.0067
...
where
function M = RajRaj()
xpr = '(?<=kN)[ 0-9\.\+\-e\r\n]++(?=(Data Header:)|(\s*$))';
str = fileread( 'RajRaj.txt' );
cac = regexp( str, xpr, 'match' );
M = nan( 11, 3, length( cac ) );
for jj = 1 : length( cac )
M( :, :, jj ) = str2num( cac{jj} );
end
end
&nbsp
I believe there is a reason why Matlab doesn't include a good functions to read a file with many blocks of numerical data.
  1 个评论
Raj Raj
Raj Raj 2014-10-27
Hi Thanks for attention and time to respond. Here comes an issue after dealing with large data file.
My data is in chunks of 11 lines. However, by some reason machine wrote down a data block whose size is 8. However, the "textscan" with chunks data option also tries to match format of data. Hence, I am saved and read the second chunk of data whose size is 8. Here comes the trouble in reading the rest of the data.
The problem is in dealing with the 'Headerlines' option in textscan. Is there any way to tell textscan to go one line backwards ?
Appreciate your inputs. Thanks in advance.

请先登录，再进行评论。

更多回答（2 个）

Geoff Hayes
Geoff Hayes 2014-10-25
Raj - presumably, each block of 8 lines that you wish to ignore is of the format
<empty line>
Data Header:
Time: 100.16064 Sec 10/8/2012 3:32:17 PM
Data Acquisition: T
Station Name: default
Test File Name:
Axial Axial Axial
segments mm kN
So there is a blank/empty line followed by seven lines with text. Each of these blocks is then followed by 11 lines that you wish to keep.
I stepped through code (using the debugger) and noticed that the first iteration of your loop worked fine - three columns of 11x1 was read successfully. However, subsequent reads failed with either nothing read, or only 4x1 columns, etc. I repeated the above and once the first block was successfully read in, I typed the following in the Command Window to see the contents of the current line
K>> fgetl(fileID)
ans =
''
which makes sense if the next line is blank. I then repeated it twice more
K>> fgetl(fileID)
ans =
''
K>> fgetl(fileID)
ans =
Data Header:
So there seemed (for some reason) to be two blank lines between each block, rather than the expected one. That suggests a problem with the format specification string which you have initialized to
formatSpec = '%d %f %f';
If I were doing this, reading in data line by line, I would add the newline "character" to the end of the format as
formatSpec = '%d %f %f\n';
And that seems to be what was missing. Using your above data as an example, both blocks were read in successfully.
Just change your formatSpec to the above, and try again.
  1 个评论
Raj Raj
Raj Raj 2014-10-27
Hi Thanks alot for your time. I tried but not able to reproduce the result as your mentioned. Instead I used following code to get the result:
fileID = fopen('testdata.dat');
formatSpec = '%f %f %f';
clear C
k = 0;N = 11;%Block size
while ~feof(fileID)
k = k+1;
if k==1,
C(k,1:3) = textscan(fileID,formatSpec,N,'HeaderLines', 8);%'CommentStyle','##',
else,
C(k,1:3) = textscan(fileID,formatSpec,N,'HeaderLines', 9);%'CommentStyle','##',
end
end
fclose(fileID)

请先登录，再进行评论。

Gabriel Felix
Gabriel Felix 2020-5-24
I had to use \n at the end of each line. Without it I couldn't make textscan() work properly, even thoug the "HeaderLines" was configured according to the text file lines. This was the only solution I found after struggling with the code for an intire day.
This was the text:
!
!
! alfa (graus) = 5.0
!
! Id. x/s z/s alfai cl c*cl/cmed cdi cmc/4
! (graus)
1 .246 .050 -1.209 .255 .332 .00538 .0170
2 .292 .150 -1.098 .259 .319 .00496 .0545
3 .339 .250 -.925 .254 .297 .00410 .0944
4 .385 .350 -.741 .243 .268 .00315 .1341
5 .432 .450 -.561 .227 .235 .00223 .1714
6 .479 .550 -.393 .206 .199 .00141 .2034
7 .525 .650 -.238 .181 .163 .00075 .2266
8 .572 .750 -.101 .152 .126 .00027 .2362
9 .619 .850 .014 .116 .089 -.00003 .2236
10 .659 .938 .103 .074 .052 -.00013 .1693
!
! CL asa = .208
! CDi asa = .00258
! e (%) = 88.9
! CMc/4 asa = .1339
My code:
%! alfa (graus) = 5.0
P = textscan(fid,'! alfa (graus) = %f','Delimiter',' ','MultipleDelimsAsOne',true,'headerLines',2,'CollectOutput',1);
alpha(1) = P{1};
%! CL asa = .208
P = textscan(fid,'! CL asa = %f\n','Delimiter',' ','MultipleDelimsAsOne',true,'CollectOutput',1,'headerLines',4+n);
CL(1) = P{1};
%! CDi asa = .00258
P = textscan(fid,'! CDi asa = %f\n','Delimiter',' ','MultipleDelimsAsOne',true,'CollectOutput',1,'headerlines',0);
CDi(1) = P{1};
%! CMc/4 asa = .1339
P = textscan(fid,'! CMc/4 asa = %f','Delimiter',' ','MultipleDelimsAsOne',true,'CollectOutput',1,'HeaderLines',2);
Cmc4(1) = P{1};

类别

在 Help Center 和 File Exchange 中查找有关 Text Files 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by