I really don't understand what you are trying to do. You seem to be mixing up symbolic and numeric variables. If you want to use the genetic algorithm, then don't use symbolic variables (do not call syms or matlabFunction).
I don't know what your reg1 and reg2 functions or arrays are, so I cannot help you debug your code any further. And I do not see an objective function to minimize, unless it is your f expression.
If you want to use the problem-based approach, you need to create an optimization problem and initial points as shown in the documentation. I suggest that you read Problem-Based Optimization Workflow. Then call solve, which will probably call fmincon. If you want to use ga, then call solve with the syntax
sol = solve(prob,x0,Solver="ga")
Good luck,
Alan Weiss
MATLAB mathematical toolbox documentation