"Is there a way to only replot the changed point and the curve around it,"
Yes: you just need to change the XData and YData values of the line object:
Forget about high-level commands like plot if you want to have this level of control, the solution is to simply create and manipulate graphics object properties:
I strongly recommend that you obtain and refer to every object using explicit handles (i.e. do NOT rely on gcf, gca, etc), e.g.:
lnh = plot(..);
..
set(lnh, 'XData',[..], 'YData',[..]);
If you are using a MATLAB version >=R2015b, then you can refer to object properties using dot syntax, which allows you to "only replot the changed point" exactly as you request, e.g. to change the third point on the line to 2:
lnh.YData(3) = 2;