The usual name for this is an implicit function. There is no relationship where you can solve directly for y as a function of x. Actually, in the function you wrote, it is technically possible, but you said this is not your real problem.)
Of course, this syntax is meaningless in MATLAB:
y^2 = @(x)y*x^2 +(1+x)/(y+1);
You have an implicit function. The simplest way to solve the problem is to use fimplicit to plot it.
Fxy = @(x,y) -y.^2 + y.*x.^2 + (1 + x)./(1 + y);
fimplicit(Fxy,[-3.5,3.5])
xlabel X
ylabel Y
As you can see in the plot, for ANY given x, there are multiple values of y. So it is not a single valued function, but one with multiple branches, and a singularity.
