calculation error for floor function

Question

Nishanth Kumar 2017-2-6

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/323572-calculation-error-for-floor-function

评论： Shlomo Geva 2017-10-29

Hello all, when I was trying for >>floor(2.3 * 50) , I was getting 114 where I suppose to get 115. Is this due to precision issue? and How to fix it?

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

Shlomo Geva 2017-10-29

在 MATLAB Online 中打开

Well, check this code: floor(p/65536) floor() does not handle correctly uint32 values of p > 4294934528 which is well within uint32 range. It is not even returning a value in the expected range 0 to 65535

>> p=uint32([4294934527, 4294934528, 2^32-1]);floor(p/65536)

ans =

1×3 uint32 row vector
   65535   65536   65536

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Sebastian 2017-2-6

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/323572-calculation-error-for-floor-function#answer_253500

在 MATLAB Online 中打开

If you have the Symbolic Math Toolbox, try this:

x = vpa(2.3 * 50)
x =
  115.0
x = floor(x)
x =
  115

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 2

Jan 2017-2-6

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/323572-calculation-error-for-floor-function#answer_253543

Yes, this is an effect of the limited precision of the IEEE754 floating point standard. See http://matlab.wikia.com/wiki/FAQ#Why_is_0.3_-_0.2_-_0.1_.28or_similar.29_not_equal_to_zero.3F . You find a lot of corresponding question, when you search e.g. for "faq 6.1".