如何保持浮点/双精度算术确定性? [英] How to keep float/double arithmetic deterministic?
问题描述
如果我们使用具有双精度和浮点运算的算法，我们如何保证在Python和C，x86和x64 Linux和Windows计算机以及ARM微控制器上运行时，结果是相同的?
我们正在使用一种算法，该算法使用:
- double + double
- 双精度+浮动
- double exp(double)
- float * float
在同一台计算机上，针对x86和x64 MinGW进行编译会产生不同的结果.该算法进行了大量数学运算，因此任何小错误最终都会有所作为.
现在，ARM mcu实现给出的结果与x86相同，但是看到这一点后，我不确定是否正确.
编辑
在所有情况下，精度损失都不是问题，只要在所有实现中都相同
编辑2
我发现这些链接非常有用，注释中已经包含一些提示:
如果我们使用具有双精度和浮点运算的算法，如何保证在Python和C，x86和x64 Linux和Windows计算机以及ARM微控制器上运行时，结果是相同的?
通常来说，除非谨慎执行自己的FP操作，否则您不能这样做.如果您使用的是各种语言的标准运算符和库以及底层的浮点硬件，则不能保证结果在不同实现中的精确可重复性.
首先，浮点数的内部表示形式存在问题.C没有指定要使用的表示形式，即使其他所有条件都相等，这也意味着您不能依靠运行在不同实现(例如x86_64和ARM)上的同一个C程序来计算相同的结果.
实际上，如今，大多数人都使用IEEE 754浮点格式，而CPython使用底层C实现的 double 类型来支持其浮点数.但是，即使到那时，IEEE仍允许在各实现之间进行少量的改动.即使要求严格遵守IEEE规范的指令和编译选项也无法完全解决此问题.
此外，您还指定要在C和Python中同时处理 double 和 float ，但是Python没有本机类似的浮动.它的本机浮点格式(可能)对应于C double .即使操作数在数值上相等，对不同的浮点数据类型执行的运算也必然会产生不同的结果，并且这种差异会在类型转换中持续存在，例如将 double 结果转换为 float .
在(机器)代码生成级别还需要考虑其他细节，例如是否将中间结果复制到FPU寄存器中或何时复制到主存储器中(可能涉及舍入)以及操作顺序.被执行.
我们正在使用一种算法，该算法使用:
double + double双重+浮动双倍exp(双倍)浮动*浮动
如果要最小化计算值之间的差异，请先选择一种浮点数据类型，然后在各处使用它.为了使Python和C实现之间的一致性，应为 double .
您还应该考虑禁用可能更改FP操作评估顺序的所有优化.可能就是所有优化.如果C编译器中有可用选项来强制执行严格的IEEE一致性，则将其打开.
您还应该在所有相关平台上测试 exp()函数的等效性.您可能需要提供自己的实现.
无论您做什么，您都应该认识到，即使在某种算法意义上都是正确的，尽管您的各种实现都产生了不同的结果，但这本身就是结果.它告诉您有关已实现的真正的计算精度的信息.
您永远都不要忘记大多数计算机FP操作会产生近似结果，因此，即使您确实设法使所有实现产生相同的结果，也并不意味着这些结果在绝对意义上肯定比其他结果更正确.附近的FP值.如果需要数字一致性，则应根据结果的特定精度对其进行量化，以实现该精度的方式实施算法，并忽略高于所选精度的差异.
If we use algorithms with double and float arithmetic, how can we guarantee that the results are the same running it in Python and C, in x86 and x64 Linux and Windows computers and ARM microcontrollers?
We re using an algorithm that uses:
- double + double
- double + float
- double exp(double)
- float * float
On the same computer, compiling it for x86 and x64 MinGW gives different results. The algorithm makes a lot of math so any small error will make a difference in the end.
Right now the ARM mcu implementation gives the same results as the x86, but after seeing this I'm not sure whether is right or not.
EDIT
Precision loss is not a problem in this case, as long as it's the same in all implementations
EDIT 2
I'm finding these links very helpful, some hints are already in the comments:
If we use algorithms with double and float arithmetic, how can we guarantee that the results are the same running it in Python and C, in x86 and x64 Linux and Windows computers and ARM microcontrollers?
Generally speaking, you cannot do so except possibly by carefully implementing your own FP operations. If you are using the various languages' standard operators and libraries and the underlying floating-point hardware, then you cannot be ensured of exact reproducability of results across different implementations.
In the first place, there is an issue with the internal representation of floating-point numbers. C does not specify the representation to be used, and even if all else were equal, that means you cannot rely on the same C program running on different implementations (e.g. x86_64 and ARM) to compute identical results.
In practice, most everyone uses IEEE 754 floating-point formats these days, and CPython uses the underlying C implementation's double type to back its floats. Even then, however, IEEE allows for a certain small amount of variation between conforming implementations. Even directives and compilation options that request strict conformance to IEEE specifications cannot fully work around that.
Additionally, you specify that you want to handle both double and float, in both C and Python, but Python doesn't have a native analog of float. Its native floating-point format (probably) corresponds to a C double. Operations performed on different floating-point data types necessarily produce different results, even when the operands are numerically equivalent, and the difference can persist across type conversions, such as converting a double result to float.
There are additional details to consider as well, at the (machine) code-generation level, such as whether or when intermediate results are copied out of FPU registers into main memory (which may involve rounding) and the order in which operations are performed.
We re using an algorithm that uses:
double + double double + float double exp(double) float * float
If you want to minimize the differences in computed values, then start by choosing one floating-point data type, and using it everywhere. For consistency between Python and C implementations, that should be double.
You should also consider disabling any and all optimizations that might change the order in which FP operations are evaluated. That might be all optimizations whatever. If you have options available in your C compiler to enforce strict IEEE conformance, then turn those on.
You should furthermore test the equivalency of the exp() functions on all relevant platforms. You may need to provide your own implementation.
Whatever you do, you should recognize that if your various implementations produce different results despite all being correct in some algorithmic sense, then that's a result in itself. It tells you something about the true precision of the computation, as implemented.
You must never forget that most computer FP operations produce approximate results, so even if you did manage to get all the implementations to produce identical results, that doesn't mean that those results are necessarily more right in an absolute sense than other nearby FP values. If numeric consistency is a requirement, then you ought to quantify that in terms of a specific precision of the results, implement your algorithm in a way that will deliver that precision, and ignore differences at precision higher than the one chosen.
这篇关于如何保持浮点/双精度算术确定性?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
