Hi Everyone,
I search a lot in the forum without findinding a solution. And pardon my English I am French :)
My problem is the following:
I have a log file with a lot of information. The log file could be up to 500Mo (even bigger sometimes). This log file is seperated in 2 main parts. A header part which is easy and fast to retrieve info line by line, and a data part.
The data part is composed of several tasks with a table of data with text before and after.
The data table structure is the following :
1 25.870 1.000 lhc 0.000 0.000 -20.140 24.449 1.42061512
2 25.870 1.000 lhc 0.000 0.000 -20.520 24.519 1.35075912
3 25.870 1.000 lhc 0.000 0.000 -20.951 24.582 1.28833133
4 25.870 1.000 lhc 0.000 0.000 -21.434 24.638 1.23204173
5 25.870 1.000 lhc 0.000 0.000 -21.958 24.689 1.18086597
6 25.870 1.000 lhc 0.000 0.000 -22.503 24.735 1.13498198
7 25.870 1.000 lhc 0.000 0.000 -23.148 24.781 1.08854135
8 25.870 1.000 lhc 0.000 0.000 -23.741 24.824 1.04623596
9 25.870 1.000 lhc 0.000 0.000 -24.244 24.863 1.00744521
10 25.870 1.000 lhc 0.000 0.000 -24.626 24.898 0.97159033
11 25.870 1.000 lhc 0.000 0.000 -24.876 24.932 0.93839531
12 25.870 1.000 lhc 0.000 0.000 -25.010 24.962 0.90779039
13 25.870 1.000 lhc 0.000 0.000 -25.057 24.990 0.87971152
14 25.870 1.000 lhc 0.000 0.000 -25.063 25.016 0.85443812
15 25.870 1.000 lhc 0.000 0.000 -25.072 25.038 0.83238819
16 25.870 1.000 lhc 0.000 0.000 -25.115 25.056 0.81396378
17 25.870 1.000 lhc 0.000 0.000 -25.220 25.070 0.79981872
18 25.870 1.000 lhc 0.000 0.000 -25.406 25.079 0.79060410
19 25.870 1.000 lhc 0.000 0.000 -25.611 25.078 0.79173920
20 25.870 1.000 lhc 0.000 0.000 -25.936 25.068 0.80208976
21 25.870 1.000 lhc 0.000 0.000 -26.373 25.047 0.82291587
22 25.870 1.000 lhc 0.000 0.000 -26.891 25.014 0.85576164
23 25.870 1.000 lhc 0.000 0.000 -27.437 24.969 0.90124460
24 25.870 1.000 lhc 0.000 0.000 -27.928 24.910 0.96048807
25 25.870 1.000 lhc 0.000 0.000 -28.254 24.835 1.03468974
26 25.870 1.000 lhc 0.000 0.000 -28.317 24.746 1.12353854
27 25.870 1.000 lhc 0.000 0.000 -28.070 24.642 1.22847010
28 25.870 1.000 lhc 0.000 0.000 -27.662 24.552 1.31821801
29 25.870 1.000 lhc 0.000 0.000 -27.101 24.452 1.41784749
30 25.870 1.000 lhc 0.000 0.000 -26.466 24.343 1.52711338
31 25.870 1.000 lhc 0.000 0.000 -25.820 24.224 1.64568471 **
As you can see in line 31, ** appears randomly as a 6th column. This is just a part of the data it goes for thousand of lines.
I am using the following code to retrieve those data. It works fine but I have performance problem with big file. It takes too long. Do you have a solution to help me improve performances ? My problem if the interruption cause by these **. The more I have the slower it gets.
Where fid is the identication of current file opened
outFile = textscan(fid, '%s', 'Delimiter', '\n');
frewind(fid);
taskSummaryFlagOn='No. goal weight pol. rot. att. 1. comp. 2. comp. residue';
taskSummaryFlagOff='Maximum of 1. component:';
needle=strfind(outFile{1}, taskSummaryFlagOn);
rowsStartTask= find(~cellfun('isempty', needle));
needle=strfind(outFile{1}, taskSummaryFlagOff);
rowsEndTask= find(~cellfun('isempty', needle));
nbStartLine=0;nbEndLine=2;
dataSimu=cell(max(size(nbLineData)),9);
nbLineData=zeros(max(size(rowsStartTask)),1);
for i=1:max(size(rowsStartTask))
nbLineData(i)=rowsEndTask(i)-rowsStartTask(i)-nbStartLine-nbEndLine;
dataSimu(i,:)=textscan(fid,'%f %f %f %s %f %f %f %f %f','headerlines', rowsStartTask(i));
while size(dataSimu{i,1},1)~=nbLineData(i)
fgetl(fid);
buff=textscan(fid,'%f %f %f %s %f %f %f %f %f');
for j=1:max(size(buff))
dataSimu{i,j}=[dataSimu{i,j};buff{:,j}];
end
end
frewind(fid);
end
If you need more information to understand my problem, I will provide you more details.
Thanks for the time you will spend to help me :)
