Precision of rounding numbers

5 次查看（过去 30 天）
Jomar
Jomar 2023-1-10
回答： Walter Roberson 2023-1-10
I am having trouble with this one.I don't know but MATLAB answer to 1/(0.1*0.00001) is not 1000000 but 999999.999999999880000, 1000000 is displayed but when I am running it with my code it always fall on D>0 so upon investigating using fprintf('%.20f',D) I found out that its answer is not 1M but 999999-ish. I am using this for evaluating discriminant therefor I can't arrived with D==0. I am really on the edge figuring this out. Any recommendation or solution to this one? Thanks a lot!

请先登录，再回答此问题。

回答（2 个）

VBBV
VBBV 2023-1-10
编辑：VBBV 2023-1-10
Ran in:
D = 1/(0.1*0.00001)
D = 1.0000e+06
fprintf('%.2d',D) % try using %d format specifier
1.00e+06
As you are using %f it is meant for floatpoint numbers,you must use %d
Walter Roberson
Walter Roberson 2023-1-10
Ran in:
D = 1/(0.1*0.00001)
D = 1.0000e+06
fprintf('%.999g\n', D)
999999.999999999883584678173065185546875
You can see that the result is not an integer.
This is expected. MATLAB computes using IEEE 754 Binary Floating Point Numbers. Binary Floating Point numbers are unable to exactly represent 1/10 or 1/100000 -- for the same mathematical reason that no finite decimal expansion is able to exactly represent 1/3 or 1/7 .
In other words, it is not a bug in MATLAB: you are encountering a limitation that will happen any time you use numbers with a finite precision, no matter what fixed numeric base you use. If you were to use base 2520, you would be able to exactly represent 1/2, 1/3, 1/4, 1/5, 1/6, 1/7, 1/8, 1/9, 1/10 -- but you would not be able to exactly represent 1/11 .

类别

在 Help Center 和 File Exchange 中查找有关 MATLAB 的更多信息

标签

产品


版本

R2014a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by