You apparently have R2016a or earlier, so you do not have ‘automatic implicit expansion’ that was part of R2016b and later versions.
Try this:
idx = bsxfun(@lt, A, 0.15*A_max);
A(idx) = NaN;
You can combine them in one line if you like. This code illustrates the approach.
