...the following command ... doesnt work. Whats my problem?
set(AX(2),gca,'YTickLabel',[0,5,10,15,20])
Got too many handles -- you told it to work on AX(2), the RH axes already, so don't want (and can't have) the gca as well--
set(AX(2),'YTickLabel',[0:20])
Actually, you really probably want to set the tick values, not just the labels. The 'ticklabel' property writes the text on existing tick marks wherever they're located and in whatever number they already exist, either wrapping the text over if fewer text entries than ticks just ignoring extra entries if more strings than the number of ticks are passed. That is, for this purpose use
set(AX(2),'YTick',[0:20])
instead and the labels will automagically reflect the tick location values and be at the locations wanted consistent with the axis limits. The label property is really user-changeable to allow you to use other labels besides the defaults like categories or the like. One other use is to fixup the sloppy formatting in the default wherein the '0' and '1' in the default axes tick marks lack the '.0' to be consistent with the intermediary '0.1', etc. It's just a little detail that makes Matlab plots look less professional than they should... :(
Oh, one last thing...on plotyy, I generally just use
set(AX(2),'xtick',[])
to just not display the second x-axis labels at all. One still must do any scaling on both, of course, but saves just a little on the formatting, too.
ADDENDUM
I've not used the recent release of Matlab "in anger" having retired from the technical consulting gig to return to family farm in roughly 2000. At that time linkprop wasn't available in R12 but the alternate way to keep the two x-axes in synch with plotyy appears to be (after just a very brief test) to be
hAx=plotyy(... % create the handles/basic figure
hLnk=linkprop(hAx,{'xtick','xticklabel','xlim'}); % link the x-axes
Now you can just use whatever operations on the one axes object and the two will stay in synch...
datetick(hAx(1),'x','keeplimits','keepticks')
and both change together. It's kinda' handy as datetick hasn't been vectorized to accept multiple handles for some reason I don't quite follow.