a + b = 21.2
(a+sqrt(a))*(b+sqrt(b)) = 170.3
So, if we take the first equation, eliminating a as 21.2-b, then we get the relation:
((21.2-b)+sqrt(21.2-b)).*(b+sqrt(b)) - 170.3 = 0
That looks vaguely like what you have written. In the second function you wrote, it looks like you tried to eliminate b as 21.2-a. I'll just use one of those forms, recognizing the symmetry.
ff1 = @(b) ((21.2-b)+sqrt(21.2-b)).*(b+sqrt(b)) - 170.3;
Note that I used .*in the middle to vectorize the code, allowing ezplot to be happy. Why? Because you should ALWAYS plot whenever you can. Plot EVERYTHING. Think about what you see.
ezplot(ff1,[0,20])
grid on

Note there are TWO points where the curve crosses the x axis. They correspond to the values of a and b. So one solution lies at roughly 6, the other at roughly 15. If we get one of the solutions as b, then the other is a, and they will sum to 21.2.
I cannot use your code to solve this (I don't have it), so fzero must suffice.
fzero(ff1,[0,10])
ans =
6.722
fzero(ff1,[10,20])
ans =
14.478
Note that I chose points that bracket the root of interest, thus [0,10] and [10,20].
You claim that you got the SAME answer each time. The problem is as I said above. a and b are completely symmetric, in that if you start out near 0, you will ALWAYS converge to the solution at 6.722. It does not matter if you call it a or b. Either of those functions will yield the result at 6.722 if you start near zero. Likewise, either function will converge to the result at 14.478 if you start above the mid point on that curve.
