check the documentation on functions http://www.mathworks.com/help/matlab/ref/function.html. after that it'll look something along the lines of
function output = yourfunction(a,b,r,x,y)
%calculate distance from (x,y) and (a,b)
%if statements to check if
%distance is == r
%elseif distance is <r
%elseif distance is >r
%return
I do not understand why you'd need a return statement