You have
numer(0) = 0;
In MATLAB, indexing can never start from 0. You need
numer(1) = 0;
for k=1:length(b)
numer(k+1) = numer(k) + b(k) * exp(i*w*k);
end
for m=1:length(a)
denom(m+1) = denom(m) + a(k)*exp(i*w*k);
end
But after that you have a problem. You could try
H = numer ./ denom
but if length(a) and length(b) are not the same, you have a problem.
