What I would try is to calculate the distances from every point to all other points. Then I would sort those distances in order of decreasing distance. Now it looks like some outliers might be fairly close to other outliers but in general the outliers are not close to more than 3 or 4 other outliers. So for a point to be valid, meaning close to a big group of other coordinates, the 5th closest (5th smallest) distance should be closer (smaller) than some specified tolerance distance. If that distance it larger, then it's far away and an outlier. So untested code would be something like
% Extract individual x, y, z vectors from the 3D data array.
allX = xyz(:, 1);
allY = xyz(:, 2);
allZ = xyz(:, 3);
fixed_xyz = xyz; % Initialize our good output array.
largestAllowableDistance = 10; % Whatever works for you.
for k = 1 : size(xyz, 1)
thisX = xyz(k, 1);
thisY = xyz(k, 2);
thisZ = xyz(k, 3);
distances = sqrt((thisX-allX).^2 + (thisY-allY).^2 + (thisZ - allZ).^2);
[sortedDistances, sortOrder] = sort(distances, 'descend');
if sortedDistances(5) > largestAllowableDistance
% It's an outlier so remove it.
fixed_xyz(k, :) = []; % Setting to null removes that row.
else
% It's near the curve.
end
end
