how to find similiarity of peak/pattern time series?

Hi @Nirwana,

Thank you for your thoughtful comments. After reviewing your feedback, I’ve taken a broader approach to quantifying the similarity between the two time series (PE and tidal data). While cross-correlation is a useful method, I’ve expanded the analysis to incorporate additional techniques that might provide a deeper understanding of the relationship between the two series. Below is an updated version of the script that includes Dynamic Time Warping (DTW), Pearson Correlation*, and Mutual Information, which allow for a more comprehensive analysis beyond just cross-correlation.

Updated MATLAB Code:

clear all; clc; % clear workspace and command window

% --- Load Data ---
PE_struct = load('PE.mat');  % Load PE data
PE = PE_struct.PE;

tidal_struct = load('tidalfile.mat');  % Load Tidal data
tidal = tidal_struct.tidal;
timetidal = tidal_struct.timetidal;

% Convert timetidal if stored as datenum
if isnumeric(timetidal)
  timetidal = datetime(timetidal, 'ConvertFrom', 'datenum');
end

% Define PE time vector
start_date = datetime(2023, 1, 1);
end_date   = datetime(2024, 5, 31);
timePE = linspace(start_date, end_date, length(PE));

% Limit data to cutoff date
cutoff_date = datetime(2024, 5, 31);
PE_idx = timePE <= cutoff_date;
PE = PE(PE_idx);
timePE = timePE(PE_idx);
tidal_idx = timetidal <= cutoff_date;
tidal = tidal(tidal_idx);
timetidal = timetidal(tidal_idx);

% --- Extract Daily Peaks ---
winDays = 1;  % 1-day window
edges = timePE(1):days(winDays):timePE(end);

% Preallocate
PE_peaks = zeros(1, length(edges)-1);
PE_times = datetime.empty(1,0);
Tide_peaks = zeros(1, length(edges)-1);
Tide_times = datetime.empty(1,0);

% Extract PE peaks
for i = 1:length(edges)-1
  idx = timePE >= edges(i) & timePE < edges(i+1);
  if any(idx)
      [val, loc] = max(PE(idx));
      PE_peaks(i) = val;
      tmp = timePE(idx);
      PE_times(end+1) = tmp(loc);
  end
end

% Extract Tidal peaks
for i = 1:length(edges)-1
  idx = timetidal >= edges(i) & timetidal < edges(i+1);
  if any(idx)
      [val, loc] = max(tidal(idx));
      Tide_peaks(i) = val;
      tmp = timetidal(idx);
      Tide_times(end+1) = tmp(loc);
  end
end

% --- Detrend Peaks ---
PE_detr = detrend(PE_peaks);
Tide_detr = detrend(Tide_peaks);

% --- Cross-Correlation (Optional) ---
n = min(length(PE_detr), length(Tide_detr));  % align lengths
[xc, lags] = xcorr(Tide_detr(1:n), PE_detr(1:n), 'normalized');
[~, I] = max(abs(xc));
bestLag = lags(I);

% --- Dynamic Time Warping (DTW) Calculation ---
dtw_distance = dtw(PE_detr, Tide_detr);
disp(['DTW distance: ', num2str(dtw_distance)]);

% --- Pearson Correlation ---
pearson_corr = corr(PE_detr', Tide_detr');
disp(['Pearson Correlation: ', num2str(pearson_corr)]);

% --- Mutual Information Calculation ---
% Estimate the histogram of each time series
nbins = 50; % Number of bins for the histogram

% Marginal Entropies
p_pe = histcounts(PE_detr, nbins, 'Normalization', 
'probability'); 
p_tide = histcounts(Tide_detr, nbins, 'Normalization', 
'probability');

% Joint Entropy
% Joint histogram: we use hist3 but in a manual way (this will 
compute joint   
histograms)
edges_pe = linspace(min(PE_detr), max(PE_detr), nbins + 1);
edges_tide = linspace(min(Tide_detr), max(Tide_detr), nbins + 1);
[counts, ~, ~] = histcounts2(PE_detr, Tide_detr, edges_pe, 
edges_tide);
p_joint = counts / sum(counts(:)); % Normalize to get joint 
probability   
distribution

% Calculate the entropy of each distribution
H_pe = -sum(p_pe .* log2(p_pe + eps));  % eps to avoid log(0)
H_tide = -sum(p_tide .* log2(p_tide + eps)); 
H_joint = -sum(p_joint(:) .* log2(p_joint(:) + eps)); 

% Calculate Mutual Information
mi = H_pe + H_tide - H_joint;
disp(['Mutual Information: ', num2str(mi)]);

% --- Plot Results ---
figure;
subplot(2,1,1);
plot(PE_times, PE_detr, '-', 'DisplayName', 'PE'); hold on;
yyaxis right;
plot(Tide_times, Tide_detr, '-', 'DisplayName', 'Tidal');
xlabel('Time'); ylabel('Detrended Values');
legend('Location','southoutside','Orientation','horizontal'); 
grid on;
title('Daily Peaks Comparison');

subplot(2,1,2);
plot(lags * winDays, xc, '-', 'LineWidth', 1.5); grid on;
xlabel('Lag (days)'); ylabel('Correlation');
title(sprintf('Cross-correlation (Best Lag = %d days)', 
bestLag));
set(gcf, 'Position', [100, 100, 900, 350]);

Please see attached.

Explanation of Results:

1.Dynamic Time Warping (DTW):

‘179544.5271` — This distance metric quantifies how well the two time series align when allowing for shifts in time. A lower value would indicate better alignment, and this result shows a relatively high dissimilarity. DTW accounts for any non-linear shifts in the series and is useful when the patterns are similar but may be temporally misaligned.

2.Pearson Correlation: `-0.099041` — This is a measure of the linear relationship between the two series. The negative value suggests a very weak inverse linear relationship, which means that while the series may look similar, their daily peaks do not align in a clear linear fashion. Pearson correlation might not capture more complex, non-linear relationships that exist between the series.

3.Mutual Information: ‘1.6276` — This metric captures the *shared information* between the two series, even if the relationship is non-linear. A positive value (greater than 0) suggests that the two series share some common information, though the amount is moderate. This is a more general measure compared to Pearson, as it can capture complex, non-linear relationships.

How This Addresses Your Comments:

Beyond Cross-Correlation: This updated analysis extends beyond cross-correlation by incorporating DTW and Mutual Information, offering a more holistic view of the relationship between the two series. These additional methods account for both temporal misalignments (via DTW) and non-linear dependencies(via Mutual Information), which cross-correlation alone might not fully capture.

Answer to Your Query on Quantification Methods: The results from DTW, Pearson Correlation, and Mutual Information give you a multi-faceted perspective on the similarity between the two time series. Cross-correlation measures the temporal alignment of the series, while DTW gives a flexible alignment metric, Pearson gives linear correlation, and Mutual Information offers insight into shared information that might not be captured by linear methods.

These methods together provide a comprehensive analysis that helps quantify the relationship more effectively than relying on cross-correlation alone.

If you need any further explanations or modifications, feel free to reach out. I’d be happy to discuss these results in more detail!