Filter regions in data with 'obvious' irregular linear trend
5 次查看(过去 30 天)
显示 更早的评论
Hello,
I have a dataset that I am plotting but there are clear trends occurring within the data that I would like to remove. I have attached a figure of a plot of some of the data. The y variable is supposed to increase exponentially as x increases, but at some points (between 9 and 14 x values), there is a control placed on the y output, causing it to hover around certain values as x increases.
Does anyone know how I may be able to identify these regions where the data has a clear linear trend between certain x values? I would usually think to have a polynominal equation fitted on the data and then index any data points that lie outside the error margins of this fit. However, I can only fit that equation by removing these irregular trends first, so a bit of an issue there.
These linear trend regions do not usually happen in chronological sequence. For instance, there may be a few period in time where the y-output is limited along x, creating a seemingly continuous trend. Thus I cannot use the 'diff' function for instance to easily identify periods when y-output varies by a specified amount. Hope that makes sense.
Feel free to ask me if I need to explain more.
0 个评论
采纳的回答
John D'Errico
2015-8-18
编辑:John D'Errico
2015-8-18
This is more difficult to do than you might think. The eye/brain is very good at seeing patterns, even with a lot of noise.
The problem is, this is not a "region",where you have a trend, but a region where SOME subset of the data is spurious, that follows a trend. And it is not even a constant, but an apparently linear trend! I could as easily pick another region wherein some subset of that huge number of points also follows a linear trend, yet they represent perfectly good data in your eyes. What is "obvious" to you is rarely "obvious" to a computer. If that was true, we would all be replaced by computers. (Or is that happening already?)
Sorry, but the best solution might be to use your eye to detect that bad data, perhaps selecting the bad points with the mouse. My selectdata tool on the file exchange might work, although it is getting a bit old. I think it may have some problems with the new graphics engine. There are also brushing options in MATLAB itself now that might suffice for your problem.
3 个评论
John D'Errico
2015-8-18
Your brain is just SOOOO good at seeing patterns, that in some cases it even sees trends where none exist. (Were that not true, I might have saved some money in the stock market, lol.)
But making a computer do it for you? Not as easy at all. If this is a one time job, or even a rare enough event, save that mental energy and use your mouse.
更多回答(0 个)
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!