The first argument to butter is the filter order, and the number of coefficients in the transfer function vectors will be 1 less than that, so:
Fs = 100;
Fco = 10;
Fn = Fs/2;
Wn = Fco/Fn;
n = 40;
[b,a]=butter(n-1,Wn,'low');
figure
freqz(b,a,2^14,Fs)
should do what you want.
Note that this creates an unstable filter.
Are you absolutely certain that you want to use a IIR filter here? A filter order of 40 is more typical of a FIR filter, especially for a lowpass application such as this. (If you want a FIR filter instead, use the kaiserord and fir1 functions to create it.)
