plot deletes the previous content of the figure, unless you use hold on, hence your patch handle becomes invalid as soon as you create the plot. To fix:
h.xt=patch(0,0,'b','erasemode','xor');
hold on
h.pth= plot(0,0,'k','markersize',2,'erasemode','background');