如何加快数百万个对象的python实例初始化? [英] How to speed up python instance initialization for millions of objects?
问题描述
我已经定义了一个名为 Edge
的python class
,如下所示:
I have defined a python class
named Edge
as follows:
class Edge:
def __init__(self):
self.node1 = 0
self.node2 = 0
self.weight = 0
现在,我必须使用以下方法创建Edge的大约10 ^ 6至10 ^ 7个实例:
Now I have to create approximately 10^6 to 10^7 instances of Edge using:
edges= []
for (i,j,w) in ijw:
edge = Edge()
edge.node1 = i
edge.node2 = j
edge.weight = w
edges.append(edge)
我在台式机上花了大约2秒钟的时间.有什么更快的方法吗?
I took me approximately 2 seconds in Desktop. Is there any faster way to do?
推荐答案
You can't make it much faster, but I certainly would use __slots__
to save on memory allocations. Also make it possible to pass in the attribute values when creating the instance:
class Edge:
__slots__ = ('node1', 'node2', 'weight')
def __init__(self, node1=0, node2=0, weight=0):
self.node1 = node1
self.node2 = node2
self.weight = weight
使用更新后的 __ init __
,您可以使用列表理解:
With the updated __init__
you can use a list comprehension:
edges = [Edge(*args) for args in ijw]
这些可以一起节省创建对象的大量时间,大约将所需的时间减半.
Together these can shave off a decent amount of time creating the objects, roughly halve the time needed.
比较创建100万个对象;设置:
Comparison creating 1 million objects; the setup:
>>> from random import randrange
>>> ijw = [(randrange(100), randrange(100), randrange(1000)) for _ in range(10 ** 6)]
>>> class OrigEdge:
... def __init__(self):
... self.node1 = 0
... self.node2 = 0
... self.weight = 0
...
>>> origloop = '''\
... edges= []
... for (i,j,w) in ijw:
... edge = Edge()
... edge.node1 = i
... edge.node2 = j
... edge.weight = w
... edges.append(edge)
... '''
>>> class SlotsEdge:
... __slots__ = ('node1', 'node2', 'weight')
... def __init__(self, node1=0, node2=0, weight=0):
... self.node1 = node1
... self.node2 = node2
... self.weight = weight
...
>>> listcomploop = '''[Edge(*args) for args in ijw]'''
和时间:
>>> from timeit import Timer
>>> count, total = Timer(origloop, 'from __main__ import OrigEdge as Edge, ijw').autorange()
>>> (total / count) * 1000 # milliseconds
722.1121070033405
>>> count, total = Timer(listcomploop, 'from __main__ import SlotsEdge as Edge, ijw').autorange()
>>> (total / count) * 1000 # milliseconds
386.6706900007557
那快将近2倍.
将随机输入列表增加到10 ^ 7个项目,时间差保持不变:
Increasing the random input list to 10^7 items, and the timing difference holds:
>>> ijw = [(randrange(100), randrange(100), randrange(1000)) for _ in range(10 ** 7)]
>>> count, total = Timer(origloop, 'from __main__ import OrigEdge as Edge, ijw').autorange()
>>> (total / count)
7.183759553998243
>>> count, total = Timer(listcomploop, 'from __main__ import SlotsEdge as Edge, ijw').autorange()
>>> (total / count)
3.8709938440006226
这篇关于如何加快数百万个对象的python实例初始化?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!