加速很长的 python 代码，该代码仅由于单个块而缓慢 [英] Speed up a long python code that proves to be slow only due to a single block

问题描述

我们有一个巨大的空间，充满了大量粒子 (~ 10^8)，具有已知的质量数组 ('HI_mass')、3d 位置 ('HI_position') 和一些有趣的分数 ('HI_fraction').在这个体积中还有一些假想的球体(~10^3)具有不同但已知的质量数组('mass_data')、位置('position_data')和大小('radius_data').

We have a huge volume in space filled with lots of particles (~ 10^8) with known array of masses ('HI_mass'), 3d positions ('HI_position'), and some interesting fraction ('HI_fraction'). There is also some imaginary spheres (~10^3) with different but known array of masses ('mass_data'), positions ('position_data'), and sizes ('radius_data') in this volume.

我们希望将每个假想球体中的所有气体粒子(对气体质量的贡献很小)相加，以得出上述每个假想球体的gas_mass".一旦我们获得了每个球体的质量，我们就可以计算一些称为sigma_HI"的数量.如果这个数量高于某个阈值，那么我们会以 {id:mass} 字典的形式跟踪该球体的个体质量，以便稍后将其用于进一步计算.第三个块需要永远在整个代码的上下文中运行，该代码很长且未包含在内；我只复制了被证明很慢的那部分代码.

We want to add up all gas particles (contributing fractionally to gas mass) in each imaginary sphere to come up with a "gas_mass" for each one of the imaginary spheres mentioned above. Once we have this mass for each sphere, then we calculate some quantity called "sigma_HI". If this quantity is above some threshold value, then we keep track of that sphere's individual mass in the form of a dictionary of {id:mass} to later use it for further calculations. Third block is taking forever to run in the context of the entire code which is very long and not included; I only copied that portion of the code that proves to be slow.

import numpy as np

enclosing_circles_gas = {}
#info of imaginary spheres where *_data are stored arrays based on some random (otherwise positive integer) ids
for id, position, radius, mass zip(id_data, position_data, radius_data, mass_data):  

    if (mass >= low_mass_cutoff):
        for i in np.where(HI_fraction > 0):                            # HI_fraction is a 1d array
            gas_mass = 0
            if (np.linalg.norm(HI_position[i] - position) <= radius):  # HI_position and position: 3d array of particles and single sphere vector
                gas_mass += HI_mass[i]*HI_fraction[i]                  # HI_mass and HI_fraction are 1d arrays of particles and their fractions
            if (gas_mass/mass >= 1.0e-6):
                enclosing_circles_gas[id] = float('{:.4f}'.format(mass))

我的问题是:如何用C++改造python中这个很慢的块来加速整个代码?

My question is: How to use C++ to transform this very slow block in python to speed up the entire code?

我尝试过的事情:将嵌套循环更改为列表推导式(但这仍然很慢)

Things I have tried: Changing the nested loop into a list comprehension (but this is still slow)

if (mass >= low_mass_cutoff):
        gas_mass = sum( HI_mass[i]*HI_fraction[i] for i in np.where(HI_fraction > 0)[0] if (np.linalg.norm(HI_position[i] - position) <= radius))
        if (gas_mass/mass >= 1.0e-6):
            enclosing_circles_gas[id] = float('{:.4f}'.format(mass))

推荐答案

您应该找到一种方法来记录循环各个部分的计时.但是，如果是字符串到字符串的双倍转换，那么以下从数字中截断数字而不是正确舍入的方法呢?

You should find a way to record timing of the individual parts of your loop. But in case it is the string to double to string conversion, what about the following way of truncating digits from the number instead of correct rounding?

import math

def trunc_digits(x, digits):
    d = math.log10(abs(x)) - digits
    d = int(math.ceil(d))
    d = math.pow(10, d)
    m = math.fmod(x, d)
    x = x - m
    return x

x = 9.87654321e-6
print (x, "->", trunc_digits(x, 4))
y = 9.87654321e6
print (y, "->", trunc_digits(y, 4))

a = -1.87654321e-6
print (a, "->", trunc_digits(a, 4))
b = -1.87654321e6
print (b, "->", trunc_digits(b, 4))

这篇关于加速很长的 python 代码，该代码仅由于单个块而缓慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！