precision problem ? why ans for 10 ^ 5 * 0.0633 is 6.3299e+03
1 次查看(过去 30 天)
显示 更早的评论
anyway to fix this ?
>> 10 ^ 5 * 0.0633
ans =
6.329999999999999e+03
>> 10 ^ 6 * 0.0633
ans =
6.329999999999999e+04
0 个评论
回答(2 个)
KALYAN ACHARJYA
2021-2-26
编辑:KALYAN ACHARJYA
2021-2-26
>> 10 ^ 5 * 0.0633
ans =
6.3300e+03
>> format shortG
>> 10 ^ 5 * 0.0633
ans =
6330
See the other Format options, you can set the different format the output display
1 个评论
Steven Lord
2021-2-26
Using a different display format masks the issue.
andy, can you post for me the exact decimal value of one divided by three? The only characters you're allowed to use to write that answer are the digits 0 through 9 and a single . for a decimal point. You're not allowed to say that the answer is anything repeating; you must write all the decimal places of your answer explicitly.
Now multiply the answer you gave me by three. The result will not be equal to one. If you had been able to write an infinite number of decimal places it would have been, but you didn't. You rounded off one divided by three and that roundoff error affected the answer.
James Tursa
2021-2-26
编辑:James Tursa
2021-2-26
Normal floating point arithmetic effects. See this link:
The root issue is that 0.0633 cannot be represented exactly in IEEE double floating point format, and downstream calculations using this will be affected as a result. Calculations that mathematically should result in a known value often do not. You need to write floating point code that is tolerant of these effects. E.g., this is the closest number to 0.0633 that IEEE double precision can represent exactly
>> num2strexact(0.0633,'fixed')
ans =
'0.0632999999999999951594276126343174837529659271240234375'
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Logical 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!