To extract only the upper triangular numbers into a column vector, you could use:
Aupper = A(triu(ones(size(A)))==1);
I think it is easier to find all duplicates than just the 'first' one. (How do we even define "first"?). However, assuming we want to progress through A column-wise, we could use:
B = unique(Aupper,'stable');
first_duplicate = Aupper(find(Aupper(1:length(B)) ~= B, 1))
The first line finds unique values in the order they appear. The second line finds the first place where the input vector and the 'uniques' are different (i.e. the first duplicate) and prints out that value.