for i=1:m-1
for j=i+1:m
t5(i,j)=fHammingDist(data(:,i),data(:,j));
b5=sum(t5)/(m-1);
end
end
Notice that you calculate b5 in each iteration of the inner for loop, overwriting the previous value completely, but you do not use the value inside the loops. There is therefore no point in calculating it inside the loops. It would make more sense to calculate it after the end of the nested loops.
You appear to be calculating the lower triangle of distances, but when you form b5 you do not fill in the upper triangle first by symmetry, so the later columns are going to be totalling fewer and fewer entries. It is not obvious that is what is desired.