So the general thing is this... Keep track of the current state outside the loop:
line_num = 0; % Current CSV line number
run_num = 0; % Current run number
ncols = 0; % Number of columns in current run
nrows = 0; % Number of rows in current run
runs = []; % Tracks actual run numbers (if they are not 1,2,3,4...)
rundata = {}; % The data for each run
I would use regexp to get the actual run number
while ~feof(fid)
current_line = fgetl(fid);
line_num = line_num + 1;
% Detect a new run
runtoks = regexp(current_line, '^Run #(\d+)', 'tokens');
if iscell(runtoks) && numel(runtoks) == 1
run_num = str2double(runtoks{1});
runs(end+1) = run_num;
rundata{end+1} = [];
ncols = 0;
nrows = 0;
continue;
end
if run_num == 0
fprintf('Warning: skipping line %d - no run header\n', line_num);
continue;
end
% First time round for each run, detect number of columns.
if ncols == 0
ncols = sum(current_line == ',') + 1;
rundata{end} = zeros(0,ncols); % yes, a 0-by-ncols matrix
end
% Parse row of data from line, and check for sanity
row = textscan( current_line, '%d', 'delimiter', ',' );
if numel(row) ~= ncols
fprintf('Column count mismatch on line %d (run %d)\n', line_num, run_num);
continue;
end
% Append the row to the current run data
nrows = nrows + 1; % (*) Note: see discussion
rundata{end}(nrows, :) = row;
end
Note (*): This could be pretty slow, because we're expanding the matrix for each run one row at a time. You can improve this in a similar fashion to how C++ vectors work: preallocate by some size (1000 rows), and grow the matrix by a constant or exponential amount when necessary:
if size( rundata{end}, 1 ) < nrows
rundata{end}(end+1:end+1000, :) = 0;
end
In the case of doing the above, you would also need to store the size of each run in an array (like I do with the runs array), and trim them to size afterwards.
Beware, I just typed this code straight into my browser. It might not compile, and may have other problems. I have not handled headers at all cos you didn't describe your data file completely, so you need to sort that out. That aside, this is the general approach I would use to read arbitrary data sets unless I needed something really special.
