If I understood correctly:
v = [1 2 3 4 5 6 5 4 3 2 1 2 3 4 5 6]; %demo data
threshold = 4;
windowsize = 5;
abovethreshold = v > threshold;
inwindowabovethreshold = conv(abovethreshold, ones(1, windowsize), 'same')
If you were using R2016a or later, you could use movsum instead of the awkward convolution.
