In the below, the assignment to S1 inside isunix() is just to load in some data since I do not have your file. In your actual code, you would use the fileread() to load data into S1 instead.
Afterwards, you would not use line numbers: you would index the mats1 cell array according to which matrix number you wanted.
filename = 'first_file.txt';
if isunix()
S1 = sprintf(' RotmA \n--------------------------\n0.999985 0.000046 0.005519\n0.000046 0.999863 -0.016557\n-0.005519 0.016557 0.999848\n___________________________\n0.999985 0.000046 0.005519\n0.000046 0.999863 -0.016557\n-0.005519 0.016557 0.999848\n')
else
S1 = fileread(filename);
end
blocks = regexp(S1, '(?<mat>(-?[\d.]+\s*){8}-?[\d.]+)', 'names');
mats1 = arrayfun(@(S) sscanf(S.mat, '%f', [3 3]), blocks, 'uniform', 0)
celldisp(mats1)
