Hi, You can solve the equation without using symbolic math. The answer can be find in a numerical way:
H = @(w) 0.1/(1 - 0.9*exp(-1i*w));
F = @(w) (abs(H(w))-1/sqrt(2))^2;
Wo = fminbnd(F,0,pi);
As the function H has a period of two times pi, there are infinite solutions: Wo+2*pi, Wo+4*pi, ....
If you really need to use symbolic math, the following sentences give the right solution in Matlab '9.4.0.885841 (R2018a) Update 3'
syms w real;
H = 0.1/(1 - 0.9*exp(-1i*w));
Sol = double(solve(H'*H == 1/2,w));
Sol(Sol < 0) = [];
