The initial problem is coming from line 36:
Lm=((kc.^2).*(L1+L2)+sqrt((k2^4).*((L1+L2).^2)+4*(kc.^4)*(1-(kc.^2)).*L1.*L2))/(2.*(1-(kc.^2)));
in this part of the calculation:
4*(kc.^4)*(1-(kc.^2))
kc is 1x11 so you can multiply kc*kc, you have to multiply them element by element
Lm=((kc.^2).*(L1+L2)+sqrt((k2^4).*((L1+L2).^2)+4*(kc.^4).*(1-(kc.^2)).*L1.*L2))/(2.*(1-(kc.^2)));
You have similar problems in the next line though.
