1) input[0] should contain the function handle. If you need to create a function handle from within the mex function, you can use mexCallMATLAB to call the MATLAB function str2func.
2) input[2] should have the same internal memory layout for the data as a regular MATLAB variable, hence it should be column major (like Fortran), not row major (like C/C++).
3) You need to create a function at the MATLAB level that can be called by fmincon. Let's assume for the moment that your objective function C++ code is within the same mex function that we are discussing (i.e., the one that is calling mexCallMATLAB to subsequently call "fmincon". Then you could create a separate m-file with the appropriate argument signature for fmincon. This m-file could in turn call your original mex function (i.e., a recursive call) but with a special additional input flag that tells the mex function it is being called from fmincon. That way the mex function can call the C++ objective function and return the result to fmincon. If the objective function depends on other inputs besides the arguments (i.e., some global variables or settings), you can have those at the top level of your C++ code so that they remain static and accessible from all the recursive calls. If you can simply separate your objective function into a separate mex function then you can go that route also (but it won't be able to easily share non-argument data with the original calling mex function.