Somewhere in your calculation, you have the ' (conjegate transpose) operator instead of the .' (non-conjugate transpose) operator.
If you are doing linear algebra, expecially with some kinds of physics, it is not uncommon that using ' (conjugate transpose) operator is mathematically correct. For example, you might have
W'*X*W
and if so then mathematically the conjugate transpose is typically the appropriate operator there.
Now, if you happen to be working with a system in which you are certain that none of the values can be complex-valued, then you can avoid the difficulty of seeing conj() by telling MATLAB that your symbolic variables are real-valued, such as
syms theta real
I recommend that you routinely tell MATLAB that symbolc variables are real-valued if you know that to be true: when MATLAB can reason that a value is surely real-valued then it will remove the conj()
