detrend using cubic splines
显示 更早的评论
How can I detrend a time series using cubic spline interpolation? I would like to get this done over for eg., 0.2 year bins.
采纳的回答
Shashank Prasanna
2013-2-8
Venkatessh, detrending is just a process of removing long term deterministic patterns or in short trends. For example your data may look like it is exponentially increasing, but with variation. You are more interested in the variation because the exponential increase renders the data non-stationary.
It is as simple as fitting your data to the best possible curve and subtracting it from your data.
Simple example:
Generate data and get a spline fitted trend:
x = 0:100;
y = 100*randn(1,length(x))+x.^2;
xx = 0:10:100;
yy = spline(x,y,xx);
Substract the trend from your original data and plot the detrended variation sequence.
ytrend = interp1(xx,yy,x,'spline');
subplot(2,1,1)
plot(x,y,'.',x,ytrend);
subplot(2,1,2)
plot(x,y-ytrend)
18 个评论
Venkatessh
2013-2-8
Image Analyst
2013-2-8
编辑：Image Analyst
2013-2-8
Maybe I don't understand, but you're merely subsampling the data, basically throwing away the original data, and then interpolating a spline between the subsampled points. Then subtracting that from the full, original data. So since you're ignoring most of the data, you're only detrending the spline knots - the data you trained the spline with. Whatever happens in between the subsample points it totally ignored as far as calculating the trend goes. That doesn't sound right to me. See this illustration:
% Create sample data:
x = 0:100;
y = 100*randn(1,length(x))+x.^2;
% Reduce every 10th data point.
y(1:10:end) = 0.5 * y(1:10:end);
% Now we have our sample data.
% Now subsample.
xx = 0:10:100;
% Fit a spline between the subsampled points only.
yy = spline(x,y,xx);
% Subtract the trend from your original data and plot the detrended variation sequence.
ytrend = interp1(xx,yy,x,'spline');
subplot(2,1,1)
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
plot(x,y,'.',x,ytrend);
grid on;
subplot(2,1,2)
plot(x,y-ytrend, 'bd-')
grid on;
title('Spline knots are detrended, other data are not', 'FontSize', 25);
Of course I guess you could say the crazy data I used doesn't really show a trend, as in the slowly varying sense of the word. As long as your data sort of lies along the spline, like your original data did, then it would probably be good enough.
Venkatessh
2013-2-8
Shashank Prasanna
2013-2-8
IA, I agree with you. In practice I have never seen detrending with a spline which as you mention does not make much sense. I am not sure what kind of time series Venkatessh is working on, but I have never seen detrending using nonparametric regression methods such as splines. You either detrend using an FIR filter or parametric regression which is intuitive. I am however referring to this.
Venkatessh, if you could provide your area of study and the application, we could assist you in a more formal solution.
Image Analyst
2013-2-8
Maybe you might try my suggested approach or even take a look at this: SLM, or at the Curve Fitting Toolbox.
Venkatessh
2013-2-8
@Benji,
but I have never seen detrending using nonparametric regression methods such as splines.
Spline smoothing can be posed as a linear parametric regression, where the unknowns are the coefficients of a linear combination of spline basis functions, e.g.,
My Answer used a similar formulation, though without the regularization, and valid only for equi-spaced time samples.
@Venkatessh
As per your discussion, I need to be very careful in discarding the data. Infact, there is always a problem of handing the uncertainties associated with each variable.
It's becoming a bit confusing whether you have the answer you need. You accepted Benji's suggestion even though we now agree that it discards the data, something you don't want to do. In your response to my Answer, you also seemed to say that it served your needs, but now you say you have an unequally spaced time series... Do you have what you came for, or don't you?
Venkatessh
2013-2-8
Matt J
2013-2-8
I don't know what you mean by a spline without control points. Knots & control points are what make a spline a spline.
Venkatessh
2013-2-8
Shashank Prasanna
2013-2-8
@Matt J, I may not entirely agree with your definition under the accepted definition of parametric regression. Splines come under a general class of non-parametric regression models although each piecewise polynomial is parametrically fitted. This is the reason I phrased specifically as "nonparametric regression methods such as splines"
Several literature categorize the method as below:
Not to argue that each piecewise polynomial is parametrized.
There are more flexible spline fit tools that let you place the spline knots unevenly. ImageAnalyst pointed you to some. There are many more on the FEX
This one looks particularly straightforward, does not require toolboxes, and is reasonably popular
I haven't used it myself, though.
Venkatessh
2013-2-8
@Benji,
It's a little murky to me why your refs view spline fitting as nonparametric when the spline basis functions, e.g., in (1.6) in the .pdf, aren't derived from the data in any way.
But regardless, going back to your original point, why would splines (parametric or no) not be a sensible tool for determining a trend? They constitute a family of curves that can be fit to a function just like any other.
Shashank Prasanna
2013-2-8
编辑：Shashank Prasanna
2013-2-8
@Matt J
A collection of local models is non-parametric, such as the one summarized in 1.6. Since it can exploit a global trend in a set of local models it can characterize some behavior in the data better than a single global model. Splines fall into this framework and is non-parametric and an accepted categorization.
I mentioned "but I have never seen detrending using nonparametric regression methods such as splines." I didn't discuss the sensibility of using this tool at all, since I haven't come across it being used. It may very well be the best tool, but I haven't seen it in the applications I have worked on and is exactly what I wanted to convey in my comment.
I'll assume that "a collection of local models" means "basis functions with localized support" and "exploit a global trend" means "capable of bending as needed to fit different regions of the curve."
So the only thing that distinguishes parametric modeling from non-parametric modeling is locality, i.e, that non-parametric offers more region-by-region local flexibility over the fit? Seems like peculiar distinction. What happens, for example, if you fit using only 1 or 2 basis functions. Then you're using the same fitting family, but you have much less local control...? But if that's the accepted terminology, I guess that's what it is.
Shashank Prasanna
2013-2-8
Honestly, I don't know if there is a definition written on stone somewhere east of here, but that is atleast the accepted convention. All the following methods fit into the non-parametric model framework: Splines, Neural Networks, SVM (and other kernel estimators), regression trees etc
Anyway this made for a nice discussion. I wish this topic wasn't embedded in another post, would loved to have see more responses.
Image Analyst
2013-2-8
I'm not sure what "I need to be very careful in discarding the data. " means. Of course any type of fitting, regression, or detrending is going to discard data. Even though the data may go into creating the fit, eventually the data is going to be discarded and replaced by the fitted/estimated/smoothed values. If you didn't want to do that, you'd just keep your original data. So you are going to discard data and replace it - the only issue to decide is how much to smooth the data.
更多回答（2 个）
Here's an example of how to do a cubic spline regression using interpMatrix. You would have to choose how finely you wanted to space the control points, which would affect the stiffness of the spline fit. In the example, the control points occur every 9 samples. After obtaining the 'trend' you would of course subtract it off the original time series to detrend it.
s = @(t) cos(2*pi*t).*exp(-abs(2*t))+ 2; %timeseries to fit
cubicBspline = @(t) (t>-1 & t<1).*(2/3 - t.^2 +abs(t).^3/2) +...
(abs(t)>=1 & abs(t)<2).*((2-abs(t)).^3/6);
tCtrlPts=linspace(-1.2, 1.2,9); %CtrlPts sample locations on t-axis
dtCtrlPts=tCtrlPts(2)-tCtrlPts(1);
tFine=linspace(-1.2, 1.2,81); %Fine sample locations on t-axis
dtFine=tFine(2)-tFine(1);
timeseries=s(tFine(:));
%create regression matrix
SampRatio=round(dtCtrlPts/dtFine); %Sampling ratio
kernel=cubicBspline(-2:1/SampRatio:2 );
nCtrlPts=length(tCtrlPts);
A=interpMatrix(kernel, 'max', nCtrlPts, SampRatio, 'mirror');
%%Do the fit!!!
trend = A*(A\timeseries);
plot(tFine,timeseries,tFine,trend,'*-');
legend('Time Series','Fitted Trend')
Image Analyst
2013-2-8
0 个投票
Because a spline is an interpolation rather than a regression, and so it goes through all the points, I don't see how it could detrend. Why not use detrend() or sgolay()?
类别
在 帮助中心 和 File Exchange 中查找有关 Spline Postprocessing 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
