Since you didn't provide info about your system. here is an example that you can study. However, I don't think that miniming the error variance works.
In the example, if you let 
, then the variance 
.
, then the variance .k0 = 1;
k = fminunc(@costfun, k0)
s = tf('s');
Gp = 1/(s*(s + 5)*(s + 10));
Gc = k*((s + 4)*(s + 0.1))/((s + 7.3)*(s + 0.01));
Gcl = feedback(Gc*Gp, 1);
t = linspace(0, 10, 1001);
step(Gcl, t)
function J = costfun(k)
s = tf('s');
Gp = 1/(s*(s + 5)*(s + 10));
Gc = k*((s + 4)*(s + 0.1))/((s + 7.3)*(s + 0.01));
Gcl = feedback(Gc*Gp, 1);
t = linspace(0, 10, 1001);
y = step(Gcl, t);
err = 1 - y;
% J = var(err);
J = trapz(t, err.^2);
end
