When the code invokes those cell arrays, what are you expecting will be returned?
{@(x,y,z)x+y+z-3;x^2+y^2+z^2-5;exp(x)+x*y-x*z-1}
means that you want to construct a cell array. The first element of the cell array is to be the handle to an anonymous function that expects three inputs and evaluates the sum of the inputs minus 3. The second element of the cell array looks in the current workspace for variables x, y, and z, in order to evaluate x^2+y^2+z^2-5 and drop the numeric result of the calculation into the cell array; this would fail if any of x, y, or z did not exist in the workspace that the cell array was being constructed in. The third element of the cell array looks in the current workspace for variables x, y, and z, in order to evaluate exp(x)+x*y-x*z-1 and drop the numeric result of the calculation into the cell array; this would fail if any of x, y, or z did not exist in the workspace that the cell array was being constructed in.
If all parts succeeded then the first parameter, f, would receive a cell array in which the first element was an anonymous function expecting 3 inputs, and the other two elements of the cell array were numeric values.
Then in the function, you try to index the cell array with x0, which contains [1,0,1] but 0 is an invalid index. If the x0 happened to contain only values that were 1, 2, or 3, then the indexing would work, returning a cell array, which you would then try to operate on with the * operator, which is not defined for cell arrays.
