浮动正在转换我的价值观 [英] Float is converting my values

问题描述

我有一个方法来测试一个值是否在字段允许的范围内。如果它在范围之外，则返回null，如果内部返回值。

  internal float？ ExtractMoneyInRangeAndPrecision（string fieldValue，string fieldName，float min，float max，int scale，int lineNumber）
 {
 float returnValue; 
 
 //检查是否有效浮点数
 if（float.TryParse（fieldValue，out returnValue））
 {
 //检查是否在范围
 if（returnValue> = min&& returnValue< = max）
 {
 int decPosition = 0; 
 decPosition = fieldValue.IndexOf（'。'）; $（
（decPosition == -1）|| 
（（decPosition！= -1）&&（fieldValue.Substring（decPosition，fieldValue.Length  - decPosition）.Length -1 <= scale））
）
 {
 return returnValue; 
} 
} 
} 
 
返回null; 
 
 $ / code $ / pre 
 
 
这是我的单元测试：
 
 
  [TestMethod（）] 
 [DeploymentItem（ImporterEngine.dll）] 
 public void ExtractMoneyInRangeAndPrecisionTest_OutsideRange（）
 {
 MockSyntaxValidator target = new MockSyntaxValidator（，0）; 
 string fieldValue =1000000; 
 string fieldName =; 
 float min = 1; 
 float max = 999999.99f; 
 int scale = 2; 
 int lineNumber = 0; 
浮动？ Int16RangeReturned; 
 
 Int16RangeReturned = target.ExtractMoneyInRangeAndPrecision（fieldValue，fieldName，min，max，scale，lineNumber）; 
 
 Assert.IsNull（Int16RangeReturned）; 
 
 
 
 
 
 $ b 
正如你所看到的，最大值是999999.99，但是当方法把它放在里面的时候将其改为1,000,000 
 
 
 
为什么会这样？
解决方案
  http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems  
 
 
 
简而言之，由于浮点数字代表实数，因此您分配给浮点数的数字并不总是您返回的数字。您指定的值将被转换为可用科学计数法表示的最接近的值，其幅度由基数2确定。在999999.99的情况下，可以表示为具有相同数量信号的浮点数的数字是7.6293945 * 2 17 = 999999.99504，当四舍五入为相同的信号时，数字是1,000,000.00。这可能不是确切的情况，但像这样的错误是固有的使用浮动。
 
 
 
不要使用浮点类型的情况下，数字的准确性在给定的精度水平是必需的。相反，使用小数类型，这将保留输入值的精度。
 
I have a method that tests a value is within the range allowed on fields. If it is outside the range returns null and if inside returns the value.
internal float? ExtractMoneyInRangeAndPrecision(string fieldValue, string fieldName, float min, float max, int scale, int lineNumber)
    {          
        float returnValue;

        //Check whether valid float if
        if (float.TryParse(fieldValue, out returnValue))
        {
            //Check whether in range
            if (returnValue >= min && returnValue <= max)
            {
                int decPosition = 0;
                decPosition = fieldValue.IndexOf('.');

                if (
                    (decPosition == -1) ||
                    ((decPosition != -1) && (fieldValue.Substring(decPosition, fieldValue.Length - decPosition).Length -1 <= scale))
                    )
                {
                    return returnValue;
                }
            }
        }            

        return null;
    }
Here is my unit test:
[TestMethod()]
    [DeploymentItem("ImporterEngine.dll")]
    public void ExtractMoneyInRangeAndPrecisionTest_OutsideRange()
    {
        MockSyntaxValidator target = new MockSyntaxValidator("", 0);
        string fieldValue = "1000000";
        string fieldName = "";
        float min = 1;
        float max = 999999.99f;
        int scale = 2;
        int lineNumber = 0;
        float? Int16RangeReturned;

        Int16RangeReturned = target.ExtractMoneyInRangeAndPrecision(fieldValue, fieldName, min, max, scale, lineNumber);

        Assert.IsNull(Int16RangeReturned);
    }
As you can see the max is 999999.99 but when the method takes it in it changes it to 1,000,000

Why is this?
解决方案
http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

In short, because of the way floating-point numbers represent real numbers, the number you assign to a float is not always the number you get back out. The value you specify is converted to the nearest value that can be represented in scientific notation with a magnitude determined by a base of 2. 

In the case of 999999.99, the nearest number that can be represented as a float with the same number of sig figs is 7.6293945 * 217 = 999999.99504, which when rounded to the same sig figs is 1,000,000.00. This may not be the EXACT case, but error like this is inherent in the use of floats.

Do not use floating-point types in situations where the accuracy of the number at a given level of precision is required. Instead, use the decimal type, which will retain the precision of values entered.

                        
这篇关于浮动正在转换我的价值观的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！