CPython和PyPy十进制运算性能 [英] CPython and PyPy Decimal operation performance

问题描述

我想用数百万个数据点(用小数表示)运行100k +仿真.我选择浮点数小数点而不是浮点数，以提高浮点精度并易于进行逻辑单元测试(因为 0.1 + 0.1 + 0.1 与浮点数不等于0.3....

I would like to run some 100k+ simulations with some millions of data points, which are represented as decimals. I choose decimals over floats for floating point accuracy and ease of unit testing my logic (since 0.1 + 0.1 + 0.1 does not equal 0.3 with floats...).

我的希望是通过使用PyPy加快仿真速度.但是在测试中，我遇到了PyPy根本无法很好地处理 decimal.Decimal 甚至 _pydecimal.Decimal 的问题，并且比CPython解释器(将C用于 decimal.Decimal 算术).因此，我复制/粘贴了我的整个代码库，并用 float 替换了所有 Decimal ，并且性能提高了很多:使用PyPy的速度是CPython的x60-x70倍-牺牲了的准确性.

My hope was to speed up the simulations by using PyPy. But during my testing I encountered that PyPy does not handle decimal.Decimal or even _pydecimal.Decimal well at all - and gets dramatically slower than the CPython interpreter (which uses C for decimal.Decimal arithmetics). So I copy/pasted my whole codebase and replaced all Decimals with floats and the performance increase was huge: x60-x70 times faster with PyPy than CPython - with the sacrifice of accuracy.

是否有解决方案可以在PyPy中使用小数精度来提高性能?我可以维护两个代码库: float 用于批处理运行100k模拟， Decimal 用于以后检查有趣的结果-但这承担了维护两个代码库的开销...

Are there any solutions to use Decimals precision in PyPy with the performance benefit? I "could" maintain two codebases: float for batch running the 100k simulations, Decimal for inspecting the interesting results later - but this bears the overhead of maintaining two codebases...

以下是我在 Raspberry Pi 4(Ubuntu Server 20.10，4 x 1.5GHZ ARM Cortex-A72，8GB RAM)上运行的一些简单测试，用于重现:

Here are some simple tests I ran on a Raspberry Pi 4 (Ubuntu Server 20.10, 4 x 1.5GHZ ARM Cortex-A72, 8GB RAM) for reproduction:

test_decimal.py

import time
from decimal import Decimal

start = time.time()
val = Decimal('1.0')
mul = Decimal('1.000001')
for i in range(10 * 1000 * 1000):
    val *= mul
end = time.time()
print(f"decimal.Decimal: {val:.8f} in {round(end-start,4)} sec")

test_pydecimal.py

import time
from _pydecimal import Decimal

start = time.time()
val = Decimal('1.0')
mul = Decimal('1.000001')
for i in range(10 * 1000 * 1000):
    val *= mul
end = time.time()
print(f"pydecimal.Decimal: {val:.8f} in {round(end-start,4)} sec")

test_float.py

import time
from decimal import Decimal

start = time.time()
val = float('1.0')
mul = float('1.000001')
for i in range(10 * 1000 * 1000):
    val *= mul
end = time.time()
print(f"float: {val:.8f} in {round(end-start,4)} sec")

结果

<身体>

测试	Python 3.8.6(GCC 10.2.0)	带有GCC 10.2.0的Python 3.6.9 -PyPy 7.3.1
test_decimal	5.1131秒	55.0829秒
test_pydecimal	315.4012秒	40.1771秒
test_float	2.5607秒	0.1273秒

编辑#1:

• 更新了示例(使用预先计算的乘法器，在 print 之外测量时间)和结果表:PyPy和CPython在十进制上的性能的总体比较保持不变.
• 仿真主要由具有变化值的时间序列数据的基本数学运算(加，减，乘，除)组成.

• Updated the examples (use precomputed multiplicator, measure time outside of print) and the results table: The overall comparison between PyPy and CPython performance on Decimals stays the same.
• The simulation mostly consists of the basic math operations (add, subtract, multiply, divide) over time-series data with changing values.

推荐答案

您可以使用 double-double precision (双精度-双精度)以比任意精度算术(即 Decimal)，并且比双精度(即 float )更准确.通常，双精度精度比四精度精度稍差一些，但是大多数平台通常都不支持后者.

You can use double-double precision to achieve what you want significantly faster than arbitrary-precision arithmetic (ie. Decimal) and more accurate than double precision (ie. float). Double-double precision is generally slightly less accurate than quad-precision, but the latter is generally not natively supported on most platforms.

doubledouble Python软件包实现了此功能，并且与PyPy兼容.它不支持字符串解析和格式化，但是您可以使用以下两种慢速方法来实现此目的:

The doubledouble Python package implement this and is compatible with PyPy. It does not support string parsing and formatting but you can implement this using the two slow following methods:

from decimal import Decimal
from doubledouble import DoubleDouble

def ddFromStr(s):
    hi = float(s)
    lo = float(Decimal(s) - Decimal(hi))
    return DoubleDouble(hi, lo)

def ddToStr(dd):
    return str(Decimal(dd.x) + Decimal(dd.y))

这里是使用方法:

start = time.time()
val = ddFromStr('1.0')
mul = ddFromStr('1.000001')
for i in range(10 * 1000 * 1000):
    val *= mul
end = time.time()
print(f"doubledouble.DoubleDouble: {ddToStr(val)} in {round(end-start,4)} sec")

这是我机器上的结果:

CPython:
  float: 22026.35564471 in 0.6692 sec
  decimal.Decimal: 22026.35566283 in 1.4355 sec
  doubledouble.DoubleDouble: 22026.35566283 in 11.62 sec

PyPy:
  float: 22026.35564471 in 0.011 sec
  decimal.Decimal: 22026.35566283 in 16.3268 sec
  doubledouble.DoubleDouble: 22026.355662823 in 0.1184 sec

如您所见，PyPy上的 doubledouble 软件包比CPython上的 Decimal 软件包快得多，而两者提供的精度相同(在这种情况下会被截断).

As you can see, the doubledouble package on PyPy is significantly faster than the Decimal package on CPython while the two provide equally accurate (truncated) results in this case.

这篇关于CPython和PyPy十进制运算性能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！