浮点精度 [英] floating point precision

查看:104
本文介绍了浮点精度的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我有一个用C#编写的程序,有些部分是用本地C / C ++编写的。
我使用双精度计算一些值,有时结果是错误的,因为太小的精度。经过一番调查,我发现有人将浮点精度设置为24位。我的代码工作正常,当我将精度重置为至少53位(使用_fpreset或_controlfp),但我仍然需要弄清楚谁负责将精度设置为24位在第一位。


解决方案

这是由默认的Direct3D设备初始化。您可以通过传递 <$ c $来告诉Direct3D不要错过FPU精度c> D3DCREATE_FPU_PRESERVE 标记为 CreateDevice 。还有一个与此标志等效的托管代码( CreateFlags.FpuPreserve )。



有关详情,请访问 Direct3D和FPU


I have a program written in C# and some parts are writing in native C/C++. I use doubles to calculate some values and sometimes the result is wrong because of too small precision. After some investigation i figured out that someone is setting the floating-point precision to 24-bits. My code works fine, when i reset the precision to at least 53-bits (using _fpreset or _controlfp), but i still need to figure out who is responsible for setting the precision to 24-bits in the first place.

Any ideas who i could achieve this?

解决方案

This is caused by the default Direct3D device initialisation. You can tell Direct3D not to mess with the FPU precision by passing the D3DCREATE_FPU_PRESERVE flag to CreateDevice. There is also a managed code equivalent to this flag (CreateFlags.FpuPreserve) if you need it.

More information can be found at Direct3D and the FPU.

这篇关于浮点精度的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆