Hello Sanket,
To modulate an input signal x by using QAM with the specified modulation order M and the symbol order symOrder use the qammod function in the following format.
y = qammod(x,M,symOrder)
You can use the following code corresponding to the function specified by you,
% x is the signal you wish to modulate
M = 16;
hmod = qammod(x,M,'gray');
% The default symbol order is gray, so you can remove if you don’t want to specify it explicitly
