Do not put the ode45 call inside the same function you're passing into ode45. At best you receive an error like the one you received; near worst you receive an error about the recursion limit; worst case scenario you've increased your recursion limit too high and crash MATLAB.
These lines should be written in the MATLAB Command Window or as part of a separate script or function.
A=1;
B=2;
tspan=[0 5];
y0=[0 0.01];
[t,y]=ode45(@(t,y) odefcn(t,y,A,B), tspan, y0);
plot(t,y(:,1),'-o',t,y(:,2),'-.')
These lines should be part of your odefcn function.
function dydt=odefcn(t,y,A,B)
dydt=zeros(2,1);
dydt(1)=y(2);
dydt(2)=(A/B)*t.*y(1);
end
You don't call odefcn directly. You pass it into ode45 which calls it with the input arguments ode45 deems necessary to solve the ODE.
