No doubt this can be simplified, but I think it meets your objective of finding the vertices (4) enclosing the points:
% create sample of regularly-spaced points, as per question
g = 0:0.1:1;
x = [];
y = [];
for i=1:10
x = [x i+g];
y = [y i-g];
end
ax = gca;
ax.XTick = 0:12;
ax.YTick = -2:12;
hold on;
plot(x,y,'.');
k = boundary(x',y');
[~, xMinIdx] = min(x(k));
[~, xMaxIdx] = max(x(k));
[~, yMinIdx] = min(y(k));
[~, yMaxIdx] = max(y(k));
xx = x(k);
yy = y(k);
p = polyshape([xx(xMinIdx) xx(yMinIdx) xx(xMaxIdx) xx(yMaxIdx)], ...
[yy(xMinIdx) yy(yMinIdx) yy(xMaxIdx) yy(yMaxIdx)]);
plot(p);
v = p.Vertices % output vertices (4)
Command window output:
v =
1 1
10 10
11 9
2 0
Figure window:
