- extract blocks of data (you need to find the start and stop times , here I used islocalmax to find peaks
- bandpass filter your probe signals , once you have measured the test signal frequency (to adapt the low and high corner frequencies of the filter
- you could easily add a simple test based on the filtered probe signals :
- amplitude ratio vs test signal amplitude
- if the ratio is very low => this is not really a valid output
- if the ratio is above 10% (for example) , again you can try to fit a sinus and extract the parameters (done in William's answer)
Finding peaks and valleys, and associated indexes, of time-shifted noisy sinosidal waves
53 次查看(过去 30 天)
显示 更早的评论
I want to automate a calibration process, but I'm really struggling with the code.
I have a signal that I give my instrument (second column of attached sample data), and then two probes which record the instrument's response, independently of each other (last two columns of attached sample data). The calibration signal is a series of sine waves of varying amplitudes and periods, see below:

I need to find the following for my calibrations:
-The signal value at each peak and trough
-Whether each probe produces sinosidal waves at each period/amplitude combination
-If so, the response value for each probe at its peaks and troughs (both as individual datapoints and as the mean value) and ideally the corresponding periods and amplitudes
The first is quite easy to do and the second can be done manually without too much trouble. Where I'm running into trouble with is the third step.
I tried the findpeaks function and it's quite noisy:

I did attempt to use the MinPeakHeight argument, use a high-pass filter, and smooth my probe data using moving averages, but none of those attempts worked very well. I also tried to use a Button Down Callback Function that would store values and indexes, but that ended up being super prone to user error in terms of the exact point to click on and not very fast. Solutions like this one don't work because the two sinusoidal response aren't quite fully phase shifted, they're time-shifted by an amount of time that's not consistent either across probes or over time.
Does anyone have any suggestions for functions to look into which might help me do what I want? I'm quite stuck.
0 个评论
回答(3 个)
Mathieu NOE
2025-9-24,11:17
编辑:Mathieu NOE
2025-9-24,11:23
Following my first comment above , this could be another approach
data=load('data.mat');
t=data.M(:,1); x=data.M(:,2); y1=data.M(:,3); y2=data.M(:,4);
samples = numel(t);
% figure
% subplot(211), plot(t,x,'-r.',t,y1,'-g.',t,y2,'-b.');
% grid on; xlabel('Time (s)'); legend('x','y1','y2');
% xlim([min(t) max(t)])
Fs = 1./mean(diff(t));
% define start/end time points of blocks of data
tmp = abs(detrend(x,1));
tmp = tmp -min(tmp); % meake tmp min equals 0
tf = islocalmax(tmp); % option 3 with islocalmax (you can also try with find peaks)
tt = t(tf);
env = tmp(tf);
figure
plot(t,tmp,tt,env,'*')
% find indexes of distant blocks
ind = find(diff(tt)>20);
tt_ind = tt(ind);
n_blocks = numel(ind);
for k= 1:n_blocks
tstop(k,1) = tt_ind(k); % end of k th block
tbefore = tt(tt<=tstop(k,1));
if k<2
tstart(k,1) = min(tbefore); % start of 1st block
else
tstart(k,1) = min(tbefore(tbefore>tstop(k-1,1))); % start of k th block
end
% add some time before and after (several seconds)
extra_time = 4; % seconds
tstop(k,1) = min(samples,tstop(k,1) + extra_time);
tstart(k,1) = max(0,tstart(k,1) - extra_time);
% now extract portion of data (k th block)
ind = (t>=tstart(k,1) & t<=tstop(k,1) );
ti = t(ind);
xx = x(ind);
yy1 = y1(ind);
yy2 = y2(ind);
% counts (input) test signal threshold crossing to get its frequency
threshold = mean(xx)+0.02; %
[t_pos,t_neg] = find_zc(ti,xx,threshold);
t_pos = unique(t_pos);
freq_input = mean(1./diff(t_pos))
% now create a bandpass filter
if freq_input<Fs/2
a = 0.5;
flow = a*freq_input;
fhigh = min(1/a*freq_input,Fs/2.56);
else
flow = 0;
fhigh = Fs/2.56;
end
[B,A] = butter(2,2/Fs*[flow fhigh]);
yy1m = mean(yy1);
yy1f = filtfilt(B,A,detrend(yy1,1));
yy1f = yy1f + yy1m;
yy2m = mean(yy2);
yy2f = filtfilt(B,A,detrend(yy2,1));
yy2f = yy2f + yy2m;
figure
plot(ti,xx,'-r.',ti,yy1f,'-g.',ti,yy2f,'-b.');
grid on; xlabel('Time (s)'); legend('x','y1','y2');
title(['Signal Freq = ' num2str(freq_input) ' Hz'])
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [ZxP,ZxN] = find_zc(x,y,threshold)
% put data in rows
x = x(:);
y = y(:);
% positive slope "zero" crossing detection, using linear interpolation
y = y - threshold;
zci = @(data) find(diff(sign(data))>0); %define function: returns indices of +ZCs
ix=zci(y); %find indices of + zero crossings of x
ZeroX = @(x0,y0,x1,y1) x0 - (y0.*(x0 - x1))./(y0 - y1); % Interpolated x value for Zero-Crossing
ZxP = ZeroX(x(ix),y(ix),x(ix+1),y(ix+1));
% negative slope "zero" crossing detection, using linear interpolation
zci = @(data) find(diff(sign(data))<0); %define function: returns indices of +ZCs
ix=zci(y); %find indices of + zero crossings of x
ZeroX = @(x0,y0,x1,y1) x0 - (y0.*(x0 - x1))./(y0 - y1); % Interpolated x value for Zero-Crossing
ZxN = ZeroX(x(ix),y(ix),x(ix+1),y(ix+1));
end
3 个评论
Star Strider
2025-9-24,18:02
编辑:Star Strider
2025-9-24,18:18
These data are not eqsy to work with, and there is so much noise that effective filtering is impossible. Also, the data were not uniformly sampled. I resampled them to a common sampling interval before proceeding with the processing. (With the exception of the nufft function, all signal processing techniques require the data to be uinformly sampled.) My code will work for non-uniformly sampled data, however I use the resampled data for convenience throughout.
There is no 'good' way to determine if they are sinusoidal. I did a regression on a sinusiod function and then did some statistics on it, however I am not convinced that they are reliable indicators of a good fit to the sinusoidal function.
Try this --
LD = load('data.mat')
t = LD.M(:,1); % Original Time Vector
s = LD.M(:,2); % Original Signal Vector
r = LD.M(:,[3 4]); % Original Response (Snesor) Vectors
t_stats = [mean(diff(t)); std(diff(t)); std(diff(t))/mean(diff(t))] % Statistics
Fs = 1/t_stats(1) % Approximate Sampling Frequency
[yr,tr] = resample([s r], t, Fs); % Resample To Constant Sampling Frequency (Sampling Interval)
tr_stats = [mean(diff(tr)); std(diff(tr)); std(diff(tr))/mean(diff(tr))] % Statistics
sr = yr(:,1); % Resmapled Signal Vector
rr = yr(:,[2 3]); % Resampled Response (Sensor) Vectors
% figure
% plot(tr, sr, DisplayName='Signal')
% hold on
% plot(tr, rr(:,1), DisplayName="Sensor 1")
% plot(tr, rr(:,2), DisplayName="Sensor 2")
% hold off
% grid
% legend(Location='best')
% xlabel('Time (unit)')
% ylabel('Amplitude (unit)')
[FTs1,Fv] = FFT1(sr,tr);
figure
plot(Fv, abs(FTs1)*2)
grid
xlabel('Frequency (unit)')
ylabel('Magnitude (unit)')
title('Fourier Transform of ''Signal''')
[FTs1,Fv] = FFT1(rr,tr);
figure
plot(Fv, abs(FTs1)*2)
grid
xlabel('Frequency (unit)')
ylabel('Magnitude (unit)')
title('Fourier Transform of ''Sensors''')
% sr_filt = bandpass(sr, [0.2 0.4], Fs, ImpulseResponse='iir');
[srpks,srlocs] = findpeaks(sr, MinPeakProminence=0.05);
idx1 = find(diff([1; srlocs])>200);
fliplocs = flip(srlocs);
idx12 = find(diff([numel(tr); fliplocs])<-200);
startlocs = srlocs(idx1)-2;
endlocs = [flip(fliplocs(idx12)); numel(sr)-2]+2;
figure
hp1 = plot(tr, sr, DisplayName='Signal');
hold on
hp2 = plot(tr, rr(:,1), DisplayName="Sensor 1");
hp3 = plot(tr, rr(:,2), DisplayName="Sensor 2");
hold off
grid
hxl1 = xline(tr(startlocs)-1,'--g', DisplayName='Start');
hxl2 = xline(tr(endlocs)+1,'--r', DisplayName='End');
xlabel('Time (unit)')
ylabel('Amplitude (unit)')
legend([hp1,hp2,hp3,hxl1(1),hxl2(1)], Location='best')
figure
hp1 = plot(tr, sr, DisplayName='Signal');
hold on
hp2 = plot(tr, rr(:,1), DisplayName="Sensor 1");
hp3 = plot(tr, rr(:,2), DisplayName="Sensor 2");
hold off
grid
hxl1 = xline(tr(startlocs)-1,'--g', DisplayName='Start', LineWidth=1.5);
hxl2 = xline(tr(endlocs)+1,'--r', DisplayName='End', LineWidth=1.5);
xlabel('Time (unit)')
ylabel('Amplitude (unit)')
legend([hp1,hp2,hp3,hxl1(1),hxl2(1)], Location='best')
xlim([740 890])
title('Zoomed Segment')
idxofst = 20;
% figure
% for k = 1:10:numel(startlocs)
% idxrng = max(1,startlocs(k)-idxofst) : min(endlocs(k)+idxofst,numel(tr));
% figure
% plot(tr(idxrng), sr(idxrng))
% hold on
% plot(tr(idxrng), rr(idxrng,1))
% plot(tr(idxrng), rr(idxrng,2))
% hold off
% grid
% xlabel('Time (unit)')
% ylabel('Amplitude (unit)')
% title("Epoch "+k)
% end
nsl = numel(startlocs)
fitfcn = @(b,t) b(1).*sin(b(2)*2*pi*t + b(3)*pi) + b(4)+b(5).*t;
f0 = 0.5;
dc = mean(sr);
figure
% % tiledlayout(numel(startlocs), size(rr,2), TileIndexing='columnmajor')
tiledlayout(5, size(rr,2), TileIndexing='columnmajor')
for k1 = 1:size(rr,2)
for k2 = 1:numel(startlocs)
% INDEX = [k1 k2]
idxrng = max(1,startlocs(k2)-idxofst) : min(endlocs(k2)+idxofst,numel(tr));
[rmin,rmax] = bounds(rr(idxrng,k1));
% Q3 = nnz(islocalmax(detrend(rr(idxrng,k1),1),MinProminence=0.01))
f0 = nnz(islocalmax(detrend(rr(idxrng,k1),2),MinProminence=0.01))/(tr(idxrng(end))-tr(idxrng(1)));
B0 = [rmax-rmin; f0; rand; dc; rand];
[B{k1,k2},nrmrsd(k1,k2)] = fminsearch(@(b)norm(fitfcn(b,tr(idxrng))-rr(idxrng,k1)), B0);
% Q4 = fitfcn(B{k1,k2},tr(idxrng))
% Q5 = rr(idxrng,k1)
% [rc(k1,k2),p(k1,k2)] = corrcoef(rr(idxrng,k1), fitfcn(B{k1,k2},tr(idxrng)));
[r,p] = corrcoef(rr(idxrng,k1), fitfcn(B{k1,k2},tr(idxrng)));
rc(k1,k2) = r(2,1);
pc(k1,k2) = p(2,1);
maxrr(k1,k2) = max(rr(idxrng,k1));
if rem(k2,9) == 0
nexttile
% figure
plot(tr(idxrng), sr(idxrng),'-b')
hold on
% plot(tr(idxrng), detrend(rr(idxrng,k1),1)+dc,'--m')
plot(tr(idxrng), rr(idxrng,k1),'-c')
plot(tr(idxrng), fitfcn(B{k1,k2},tr(idxrng)),'-r')
hold off
grid
title("Sensor "+k1+", Segment: "+k2)
end
end
end
sgtitle('Selected Epochs')
Bmtx = cell2mat(B);
Bmtx_1 = Bmtx(1:5,:);
Bmtx_2 = Bmtx(6:10,:);
% nrmrsd
% rc
% pc
Sensor_1 = array2table([Bmtx_1.', maxrr(1,:).', nrmrsd(1,:).', rc(1,:).', pc(1,:).'], VariableNames=["Amplitude","Frequency","Phase","Intercept","Slope","Maximum","Norm Resid","R","p"] );
Sensor_2 = array2table([Bmtx_2.', maxrr(2,:).', nrmrsd(2,:).', rc(2,:).', pc(2,:).'], VariableNames=["Amplitude","Frequency","Phase","Intercept","Slope","Maximum","Norm Resid","R","p"] );
disp(Sensor_1)
disp(Sensor_2)
function [FTs1,Fv] = FFT1(s,t)
% One-Sided Numerical Fourier Transform
% Arguments:
% s: Signal Vector Or Matrix
% t: Associated Time Vector
t = t(:);
L = numel(t);
if size(s,2) == L
s = s.';
end
Fs = 1/mean(diff(t));
Fn = Fs/2;
NFFT = 2^nextpow2(L);
FTs = fft((s - mean(s)) .* hann(L).*ones(1,size(s,2)), NFFT)/sum(hann(L));
Fv = Fs*(0:(NFFT/2))/NFFT;
% Fv = linspace(0, 1, NFFT/2+1)*Fn;
Iv = 1:numel(Fv);
Fv = Fv(:);
FTs1 = FTs(Iv,:);
end
EDIT -- Re-ran code.
.
2 个评论
Star Strider
2025-9-24,21:04
Thank you!
I'm not sure what Oshani wants. The sensors are not creating an accurate representation of the signal, so I have no idea what the calibration is going to do or what the actual objective is here. It may be that a relatively high-frequency sine is not appropriate for the instrumentation, and a step might be better.
I started working on this last night, however my computer crashed -- unusual for Ubuntu -- and since it ws late, I just gave up for the night. This took seeral hours to analyse.
William Rose
2025-9-24,8:00
You write that you want (and I have added numbers to your three desires),
1. The signal value at each peak and trough
2. Whether each probe produces sinosidal waves at each period/amplitude combination
3. If so, the response value for each probe at its peaks and troughs (both as individual datapoints and as the mean value) and ideally the corresponding periods and amplitudes
And you write "The first is quite easy to do and the second can be done manually without too much trouble. Where I'm running into trouble with is the third step."
I am glad that you can determine manually and without much trouble whether the waves are sinusoidal. That is beyond me. I would have to use function thd() or something like it to quantify sinusoidal-ness.
The interval between samples varies, which will complicate the analysis. The plots below show the three signals (top) and the sampling interval (middle). The input signal is labelled "x" and the outputs are "y1" and "y2". I will discuss the bottom panel later.
data=load('data.mat');
t=data.M(:,1); x=data.M(:,2); y1=data.M(:,3); y2=data.M(:,4);
figure
subplot(311), plot(t,x,'-r.',t,y1,'-g.',t,y2,'-b.');
grid on; xlabel('Time (s)'); legend('x','y1','y2');
subplot(312), plot(t(2:end),diff(t),'-k.');
grid on; xlabel('Time (s)'); ylabel('Sampling Interval (s)')
If the long intervals only occur between the pulses, then it would not be a big problem, but that is not the case. For example, if you zoom in on the time period 2020<=t<=2060, you will see that the long time intervals sometime occur in the middle of a sinusoidal pulse.
The bottom panel is a zoomed-in view of the first pulse. The plot shows that the signal is not sampled rapidly enough to determine if it is sinusoidal or not.
subplot(313)
plot(t,x,'-r.',t,y1,'-g.',t,y2,'-b.');
grid on; xlabel('Time'); legend('x','y1','y2');
xlim([73,76]); ylim([16.6,17.4])
The bottom panel shows 10 data points for 4 cycles of oscillation, i.e. the frerquency is 0.4 cycles per sample. If this wave is non-sinusoidal, i.e. if it has power at harmonics of the fundamental frequency, then those harmonics will be above the Nyquist frequency (0.5 cycles/sample), so they will be aliased down to lower frequencies, in the range 0 to 0.5 cycles/sample, and you will not be able to un-alias them or do a reliable analysis of the signal.
Can you resample the signals faster and at a constant rate?
To accomplish your third aim, which is to find the period and amplitude of the signal and probe responses for the sinusoidal pulses, I recommend fitting a sinusoid to the signal pulse and to the pulses recorded by the probes. There is no doubt a clever way to determine where the pulses start and end*, but for the sake of simlicity I will simply determine the pulse start and end times by inspecting the plot. Then I wil fit the data in those windows. Start and end times of first four pulses and last four pulses:
% Enter pulse start, end times, determined by inspecting the plots
Ts=[73.8,112,152,193.9,2213.8,2257.3,2308.2,2362.7];
% pulse start times (<= time of first point in each pulse)
Te=[75.6,115.6,157.4,201.1,2221,2271.7,2326.4,2391.4];
% pulse end times (>= time of last point in each pulse)
The next chunk of code is a funciton that fits a sinusoid to x,y data:
function [f,gof] = fitSinusoid(x,y)
% FITSINUSOID Fit sinusoid to data
% Model: y=a*sin(b*x+c)+d
% Make a good initial guess for parameters in order to increase
% chance of getting a good fit.
a0=(max(y)-min(y))/2; % amplitude initial guess
nzc=length(find(diff(sign(y-mean(y)))==-2));
% number of negative-going zero crossings
b0=2*pi*nzc/(max(x)-min(x)); % radian freq initial guess
c0=0; % phase initial guess
d0=mean(y); % offset initial guess
myModel=@(a,b,c,d,x) a*sin(b*x+c)+d;
[f1,gof1]=fit(x,y,myModel,...
'StartPoint', [a0,b0,c0,d0], ...
'Lower', [0, 0, 0, -Inf], ...
'Upper', [Inf, Inf, max(x)-min(x), Inf]);
[f2,gof2]=fit(x,y,myModel,...
'StartPoint', [2*a0,b0,c0,d0], ...
'Lower', [0, 0, 0, -Inf], ...
'Upper', [Inf, Inf, max(x)-min(x), Inf]);
if gof1.sse<=gof2.sse
f=f1; gof=gof1;
else
f=f2; gof=gof2;
end
end
You may be wondering why I didn't just use the built-in fittype 'sin1'. The reason is that 'sin1' does not include a mean offset, as far as I can tell.
The next chunk of code fits the signal and probe repsonses during each pulse, and displays the best-fit amplitude and frequency and r^2 on the console. It also displays plots of the data and the fits.
% Analyze each pulse
Np=length(Ts); % number of pulses
for i=1:Np
[fx,gofx] =fitSinusoid(t(t>=Ts(i) & t<=Te(i)),x(t>=Ts(i) & t<=Te(i)));
[fy1,gofy1]=fitSinusoid(t(t>=Ts(i) & t<=Te(i)),y1(t>=Ts(i) & t<=Te(i)));
[fy2,gofy2]=fitSinusoid(t(t>=Ts(i) & t<=Te(i)),y2(t>=Ts(i) & t<=Te(i)));
fprintf('Pulse %d:\n',i);
fprintf(' Signal: ampl=%.3f, freq=%.3f, r^2=%.2f.\n',...
fx.a,fx.b/(2*pi),gofx.rsquare)
fprintf(' Probe1: ampl=%.3f, freq=%.3f, r^2=%.2f.\n',...
fy1.a,fy1.b/(2*pi),gofy1.rsquare)
fprintf(' Probe2: ampl=%.3f, freq=%.3f, r^2=%.2f.\n',...
fy2.a,fy2.b/(2*pi),gofy2.rsquare)
figure
subplot(311)
plot(fx,t(t>=Ts(i) & t<=Te(i)),x(t>=Ts(i) & t<=Te(i)))
xlabel('Time'); grid on
subplot(312)
plot(fy1,t(t>=Ts(i) & t<=Te(i)),y1(t>=Ts(i) & t<=Te(i)))
xlabel('Time'); grid on
subplot(313)
plot(fy2,t(t>=Ts(i) & t<=Te(i)),y2(t>=Ts(i) & t<=Te(i)))
xlabel('Time'); grid on
end
The plots show that the fits are not always the best, but maybe with some tweaking, you can get this approach to be useful. Maybe use fmincon() instead of fit(). Also, the script reports r^2=-1.08 for pulse 6. That shouldn't happen, so it merits further investigation.
Good luck with your analysis.
1 个评论
Mathieu NOE
2025-9-24,9:01
dear all
IMHO the test signals have a too high frequency , the probe will not keep with the firsts sine bursts that are obviously beyond their frequency bandwidth
maybe you should redo the test with lower freq bursts (and that would also increase the number of samples per period )
to test the sinus like shape maybe we could also use first some bandpass filtering of the probe signals , based on the frequency measured on the input (test) signal. frequency can be measured from the "zero" crossing rate of the signal (using the mean of the signal as the threshold value) - see second subplot here
data=load('data.mat');
t=data.M(:,1); x=data.M(:,2); y1=data.M(:,3); y2=data.M(:,4);
figure
subplot(211), plot(t,x,'-r.',t,y1,'-g.',t,y2,'-b.');
grid on; xlabel('Time (s)'); legend('x','y1','y2');
xlim([min(t) max(t)])
% counts signal crossing
threshold = mean(x)+0.1; %
[t0_pos,t0_neg] = find_zc(t,x,threshold);
freq_input = [0;1./diff(t0_pos)];
% discard values below 0.05 Hz (generated by distant t0_pos belonging to separate burst)
ind = (freq_input<0.05);
freq_input(ind) = [];
t0_pos(ind) = [];
subplot(212), plot(t0_pos,freq_input,'*');
grid on; xlabel('Time (s)'); ylabel('Signal freq (Hz)')
xlim([min(t) max(t)])
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [ZxP,ZxN] = find_zc(x,y,threshold)
% put data in rows
x = x(:);
y = y(:);
% positive slope "zero" crossing detection, using linear interpolation
y = y - threshold;
zci = @(data) find(diff(sign(data))>0); %define function: returns indices of +ZCs
ix=zci(y); %find indices of + zero crossings of x
ZeroX = @(x0,y0,x1,y1) x0 - (y0.*(x0 - x1))./(y0 - y1); % Interpolated x value for Zero-Crossing
ZxP = ZeroX(x(ix),y(ix),x(ix+1),y(ix+1));
% negative slope "zero" crossing detection, using linear interpolation
zci = @(data) find(diff(sign(data))<0); %define function: returns indices of +ZCs
ix=zci(y); %find indices of + zero crossings of x
ZeroX = @(x0,y0,x1,y1) x0 - (y0.*(x0 - x1))./(y0 - y1); % Interpolated x value for Zero-Crossing
ZxN = ZeroX(x(ix),y(ix),x(ix+1),y(ix+1));
end
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 ECG / EKG 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!