The equations you are trying to solve with X = A\B are given by:
A*X = B
where from the sizes of A and B, X must be of size 6 x 4. For each column of X you have four equations and six unknowns, making a total of sixteen equations and twenty-four unknowns, so this problem is highly indeterminate. There are infinitely many possible solutions. In other words the values in A and B are insufficient to deduce completely what values X must have had. Consequently matlab arbitrarily set eight of the X values to zero. It is no surprise that these don't match your original values in X. To reconstruct X you need enough additional information to pin down eight more degrees of freedom.
