Efficient Moving Quantile Function

22 次查看（过去 30 天）
Matthew Casiano
Matthew Casiano 2020-1-11
评论： Benjamin Stucky 2023-11-6
Hi,
I am looking for thoughts on a recipe to perform a moving quantile in Matlab. Essentially I am trying to obtain the 98% value, for example, of a window that is instantaneously tracked through a large data set. I have tried looping in the quantile function, but this is extremely slow. I have had reasonable success using the sort function each time the window is moved, and then selecting the value based on the index that is 98% of the window size - see script below. But it is still several orders of magnitude slower than the built-in movmedian function. I don't see how a moving quantile should be much different then a moving median, just that the sorted value selected is not at 50%.
My thoughts are that after the initial window is sorted, an insertion sort algorithm is most appropriate since only a single value comes into the window each time the window moves 1 point. But really I am guessing at what is inside the movmedian function. I am not a programmer guy, so any advice here or examples on how to code an efficient moving quantile function is appreciated.
The example below is relatively quick with the given sampling rate, however in my example the sampling rate is 100,000 sps and takes much longer.
% Moving Quantile
fs=1000; % sampling rate (sps)
timeData=0:1/fs:40; % time (sec)
Data=rand(1,length(timeData)); % data (EU)
window=6300; % number of points used in 1 pt sliding window (pts)
dt_wnd=(window-1)/fs; % window time span (sec)
time_wnd=timeData(1)+dt_wnd/2:1/fs:timeData(end)-dt_wnd/2; % time vector, sliding window center points (sec)
Ind=floor(0.98*window); % index at 98% quantile
for j=1:length(time_wnd) % loop through all windows moving by 1 pt
SortedData=sort(Data(j:j+window-1)); % sorts data in the jth window
Val(j)=SortedData(Ind); % value at index
end
% Plotting
plot(timeData, Data)
axis([0 40 0.9 1.0])
hold on
plot(time_wnd,Val,'LineWidth',4)
hold off
title('Moving Quantile')
xlabel('time, s')
ylabel('Amplitude')
  1 个评论
Benjamin Stucky
Benjamin Stucky 2023-11-6
I am also interested in moving quantiles. The implementation of @Eike Petersen works great for smaller data sets. But I am dealing with very large time series. I found this C# reference implementation: https://aakinshin.net/posts/partitioning-heaps-quantile-estimator/ As I have no knowledge about C# and C++ wrappers, I am not able to create a mex file from this. If somebody could help to implement it, this would be nice.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Eike Petersen
Eike Petersen 2020-12-11
My guess is that your guess is pretty good. :-)
I just hand-implemented a version of this with a naive insertion sort algorithm, and it's roughly 50-80x slower than medfilt1. I'm not sure you can get much quicker without switching to C / a mex-file.
You can find it here; it works for N-D arrays, with NaNs, and with different padding strategies: https://de.mathworks.com/matlabcentral/fileexchange/84200-movquant

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Shifting and Sorting Matrices 的更多信息

标签

产品


版本

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by