Perhaps try this:
axes(handles.averSpec); % Make averSpec the current axes.
cla reset; % Do a complete and total reset of the axes.
If you want to clear just the curve, bars, surfaces, or whatever you plotted, and not the labels, tick spacing etc., then get the handle to it when you created it and then delete it
hPlotData = plot(....... whatever
% Now get rid of it.
delete(hPlotData);
