We don't know what odefun or vectorize are, but it doesn't make sense to be using varargin here. Have you tried looking at the doc page? There are some simple examples there that might help. Here's one I modified slightly to match your calling syntax above.
A = 1;
B = 2;
tspan = [0 5];
y0 = [0 0.01];
[t,y] = ode45(@odefcn,tspan, y0, [],A, B);
plot(t,y(:,1),'-o',t,y(:,2),'-.')
function dydt = odefcn(t,y,A,B)
dydt = zeros(2,1);
dydt(1) = y(2);
dydt(2) = (A/B)*t.*y(1);
end
Note that ode45 has to have the first 2 inputs be time and initial conditions. These correspond to the odefcn inputs t and y. The empty brackets are for options. I can then tack on any remaining inputs to odefcn after that in order (A, then B).
The other way is to use the calling syntax shown in the doc:
[t,y] = ode45(@(t,y) odefcn(t,y,A,B), tspan, y0);
