MATLAB uses IEEE 754 Binary Double Precision to represent floating point numbers. All floating point scheme that use binary mantissas cannot exactly represent 1/10, just like finite decimal representation schemes cannot exactly represent 1/3 or 1/7 .
IEEE 754 also defined a Decimal Double Precision representation scheme, which can represent 2.123 exactly. However, computing those values in software is much slower. The only systems I know of that implement IEEE 754 Decimal Double Precision in hardware are the IBM z90 series.
If you need a certain specific number of decimal places to be stored, then use rationals with a power-of-10 denominator.