You can run this with two functions. The first function I have designated as myMain.m and it is written as..
r0 = rand(3,3);
fsolve(@myfun,r0)
The second function myfun.m is called by the first function by passing initial guesses of r as r0. I altered the way your equation above was written and wrote the function as...
function F = myfun(r)
Bx = rand(3,1);
B = rand(3,1);
U = rand(1);
F = (U)*sqrt(-2+((3*r.^3)*(Bx/U)))-r.^3*B;
end
Change the matrices to whatever numbers you need and it should work.
Good luck.
