Apparently, none of the available functionalities of optimalleaforder does the work. So here it is a function that runs through the rows of the tree matrix generated by linkage and produces your output.
function c = binOrd(tree)
p = size(tree,1);
c = zeros(1,p+1);
k = 0;
recBin(p);
function recBin(r)
x = tree(r,1:2)-p-1;
if x(1)<=0
c(k+1) = tree(r,1);
k = k+1;
else
recBin(x(1));
end
if x(2)<=0
c(k+1) = tree(r,2);
k = k+1;
else
recBin(x(2));
end
end
end
To get your desired rightmost binary ordering, run dendogram with 'Reorder' set as the output of the binOrd function:
tree = linkage(D,'average');
myBinOrder = binOrd(tree);
dendrogram(tree,'Reorder',myBinOrder);
