There are many things here that could be causing issues:
The error you see is because the third argument to the integral function (xmax) must be a scalar. You have given T, which I assume is a matrix.
Also, you have flipped the first two arguments of fitnlm. The first should be the input matrix T, the second should be the response vector g_a.
modelfun must be a vectorized function, meaning that its parameter y is a matrix where each row corresponds to an observation, and it returns a column vector where each row is the response from that observation. You may find it easier to use a local or a nested function instead of an anonymous function.
Finally, the parameter y of modelfun is shadowed by the parameter y of the integral function. This would be another reason to use a local or a nested function, or you could simply rename one of the parameters.
