It's just what it sounds like, though if you have the Image processing Toolbox you can use the immse() function
meanSquaredError = immse(signal, referenceSignal);
fprintf('\n The mean-squared error is %0.4f\n', meanSquaredError);
If you don't have that, then subtract the two signals, square it, and take the mean. What is your reference signal that you're using?
