square = unique(square,'rows'); works to remove repeated rows, yes.
In I uderstand the ordering you want, it is to sort by y-coordinate (increasing) and where y-coordinates are equal sort by x-coordinate (increasing). You can sort square in this manner using square = sortrows(square,[2 1]);
load square
square = unique(square,'rows');
square(1:5,:)
square = sortrows(square,[2 1]); % sort by y, then by x
square(1:5,:)
