For comparing each line of a file to the corresponding line in another file for 50,000 files with 50,000 lines each, you are essentially looking to perform a pairwise comparison between two sequences at each timestep. Since "findchangepts" isn't designed for this purpose, you would need to implement a custom comparison.
Refer to the psuedocode below for an example on how this can be achieved:
% Assuming the files are named in a systematic way that allows you to match them
for t = 1:50000
% Construct file names for the current timestep
file1 = sprintf('path_to_files/timestep_%d_file1.txt', t);
file2 = sprintf('path_to_files/timestep_%d_file2.txt', t);
% Open the files
fid1 = fopen(file1, 'r');
fid2 = fopen(file2, 'r');
% Initialize an array to hold RMS values for this timestep
rms_values = zeros(50000, 1);
% Read and compare each line
for line = 1:50000
% Read lines from both files
line1 = fgetl(fid1);
line2 = fgetl(fid2);
% Convert lines to numerical arrays if necessary
% This step depends on the format of your data
data1 = str2num(line1); % Example conversion
data2 = str2num(line2); % Example conversion
% Calculate the RMS of the difference
rms_values(line) = sqrt(mean((data1 - data2).^2));
end
% Close the files
fclose(fid1);
fclose(fid2);
end
Further the RMS values can be analysed for significant changes as required to do similar analysis to "findchangepts".
Hope this helps!
