Hi Mat Fan,
Yes, vectors are supported in MEX. If you want to generate a MEX file from C or C++ code and run it in MATLAB, the C file should be written using the "C MATRIX API" and the "MATLAB DATA API" for C++.
Please refer to the following resource for guidance on writing a C or C++ source file in order to compile it into a MEX file,
In this case, it seems like there was another function with the same name but different function definitions. Make sure to use the standard gateway function. Within this gateway function, you can call your custom functions.
For more information, refer the following resources,
C++ Data API - https://www.mathworks.com/help/matlab/matlab-data-array.html
mex command - https://www.mathworks.com/help/matlab/ref/mex.html