Here's one possible solution; it first checks for criteria 2 and then goes back to verify #1. You can probably do it all in one fell swoop but finding consecutive runs can get messy so I prefer to keep that on its own. I also opted not to mark the extra groups with a NaN, because splitapply and other grouping functions insist on consecutive-integer groupings; I find it cleaner to filter those out after the fact.
% First group by the within-15-minutes criteria
dt = minutes(diff(test1.TIMESTAMP));
grp = cumsum([true; dt>15]);
% Now check that each subgroup has at least a 5-minute run of consecutives
maxrun = @(x) max(diff([1; find(minutes(diff(x)) > 1); length(x)+1]));
isgood = splitapply(@(x) maxrun(x)>=5, test1.TIMESTAMP, grp);
% Sum of each group
grpsum = splitapply(@sum, test1.mean_V, grp);
grpsum(~isgood) = NaN;
