红宝石计算浮动错了吗？ [英] Does ruby calculate floats wrong?

问题描述

这里有什么不对？ （ruby version：1.9.2p290（2011-07-09 revision 32553）[x86_64-darwin11.0.0]

  x = 523.8 
w = 46.9 
 xm = x + w 
 assert_equal w，（xm -x）#FAILS with <46.9> expected，但是<46.89999999999998> 
  




解决方案

From 浮点指南：



数字，如0.1 + 0.2加起来好一轮0.3，和
，而不是我得到一个奇怪的结果，如0.30000000000000004？

因为内部，计算机使用的格式（二进制浮点）
不能准确地表示一个数字，如0.1,0.2或0.3。

当代码被编译或解释，您的0.1已经是
四舍五入到格式的最接近的数字，这导致甚至在计算发生之前小
四舍五入错误。

阅读链接网站以获取详细信息和解决方法。

Whats wrong here? (ruby version: 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin11.0.0]

x = 523.8
w = 46.9
xm = x + w
assert_equal w, (xm - x) # FAILS with: <46.9> expected but was <46.89999999999998>

解决方案

From The Floating-Point Guide:

Why don’t my numbers, like 0.1 + 0.2 add up to a nice round 0.3, and instead I get a weird result like 0.30000000000000004?

Because internally, computers use a format (binary floating-point) that cannot accurately represent a number like 0.1, 0.2 or 0.3 at all.

When the code is compiled or interpreted, your "0.1" is already rounded to the nearest number in that format, which results in a small rounding error even before the calculation happens.

Read the linked-to site for details and ways to get around this.

这篇关于红宝石计算浮动错了吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！