优化内循环复制 [英] Optimizing Inner Loop Copy
问题描述
大家好，
我的代码里面有我的内循环：
allNew = []
for params in case：
newObj = copy（initialObject）
newObj.modify（params）
allNew.append（newObj）
返回allNew
并且副本占我执行时间的大部分（42％）。
所以，我'我想加快我的副本。我有一个明确的复制方法，它需要什么，然后返回一个新对象，但这比使用标准的lib copy.copy（）慢了很多。 。
这是'我复制的对象类：
class规则（列表）：
def __init __（self，lhs = None，rhs = None，nClasses = 0，nCases = 0）：
self.nClasses = nClasses
self.nCases = nCases
如果lhs不是None：
self.extend（lhs）
如果rhs是None：
self.rhs = tuple（）
else：
self.rhs = rhs
＃as我提到过，Rule.copy比copy.copy慢。
def copy（self）：
r =规则（自我，
self.rhs ，
self.nClasses，
self.nCases）
返回r
基本上，规则的左侧（例如，规则是& b& c -d）是
self和其他三个信息，两个整数和一个
右手边（例如，（d，5）意味着d取值5 ...所有
LHS元素也是元组）。
就优化而言，似乎我可以写一个自定义的__copy__
方法，但是一个clpython搜索（对不起，忘了给参考书签书签）
似乎表示复制所有
对象的属性的特殊用途__copy__将丢失到通用copy.copy（）。这个
看起来不太正确......想法？
下一个选项就是用C重写整个班级。我还没有之前和之后完成了C
扩展，因为大部分课程已经是内置列表，
看起来像很多工作而收益不大。 Pyrex会在这里提供帮助吗？
我不认为相关的打字信息（单独注明，我猜）
会让我走得很远......再次，在C中重新编码列表内容（手动或通过
Pyrex）不会获得任何收益（它可能会减慢速度？）。
在我看来，浅层副本（由不可变类型构建的对象）
应该能够通过内存映射加速（不知何故！）。顶级
列表/规则应该是唯一需要创建的新参考。
任何快速答案或最有可能探索的方向，非常感谢
赞赏。
问候，
马克
Mark E. Fenner写道：
>
和副本占大多数（42％）我的执行时间。
所以，我想加快我的副本。我有一个明确的复制方法，它需要什么，然后返回一个新对象，但这比使用标准的lib copy.copy（）慢了很多。 。
你是如何衡量的？在我看来，你的Rule.copy方法比copy.copy快得多
：
>> r =规则（范围（100））
shell.timefunc（r.copy）
''复制（...）36458次迭代，每次调用13.71usec''
>>来自复制导入副本
shell.timefunc（copy，r）
''复制（...）4498次迭代，每次调用111.17usec''
>>>
其中shell.timefunc为：
def _get_timer（）：
如果sys.platform ==" win32"：
返回time.clock
else：
返回time.time
返回
def timefunc（func，* args，** kwds）：
timer = _get_timer（）
count，totaltime = 0,0
总时间< 0.5：
t1 =计时器（）
res = func（* args，** kwds）
t2 = timer（）
总时间+ =（t2-t1）
计数+ = 1
如果计数1000：
unit =" usec" ;
timeper = totaltime * 1000000 / count
else：
unit =" msec"
timeper =总时间* 1000 /计数
返回％s（...）％s次迭代，％。每次调用％.2f％s ％\
（func .__ name__，count，timeper，unit）
Michael
Michael Spencer写道：
Mark E. Fenner写道：
>>
而副本占我执行时间的大部分（42％）。
所以，我想加快我的副本。我有一个明确的复制方法，它确实需要并返回一个新对象，但这比使用标准的lib copy.copy（）要慢得多。
您如何衡量？在我看来，你的Rule.copy方法比copy.copy快了很多b / b：
>> r =规则（范围（100））
>> shell.timefunc（r.copy）
''复制（...）36458次迭代，每次调用13.71usec''
>> from复制导入副本
>> shell.timefunc（copy，r）
''复制（... ）4498次迭代，每次调用111.17usec''
< snip>
Michael
迈克尔，
谢谢。我在某处错误地解释了某些东西...使用Rule.copy，程序确实更快了
。我需要查看其余的分析数据，
，看看我是否也搞砸了其他地方。
那么，如何优化Rule.copy （）？
问候，
马克
马克E. Fenner写道：
这里是我复制的对象类：
这里''有些事情可能有助于加快你的__init__
方法，因此你的复制方法：
>
class规则（列表）：
def __init __（self，lhs = None，rhs = None，nClasses = 0，nCases = 0）：
def __init __（self，lhs = None，rhs =（），nClasses = 0，nCases = 0）：
self.nClasses = nClasses
self.nCases = nCases
如果lhs不是None：
self.extend（lhs）
extend方法有什么作用？如果它很小，可能在这里内联一份副本
。
>
如果rhs为None：
self.rhs = tuple（）
else：
self.rhs = rhs
将以上4行替换为：
self.rhs = rhs
>
＃正如我所提到的，Rule.copy比copy.copy慢。
def copy（self）：
r =规则（自我，
self.rhs，
self.nClasses，
self.nCases）
返回r
HTH，
John
Hello all,
I have a code where my inner loop looks like:
allNew = []
for params in cases:
newObj = copy(initialObject)
newObj.modify(params)
allNew.append(newObj)
return allNew
and the copy is taking the majority (42%) of my execution time.
So, I''d like to speed up my copy. I had an explicit copy method that did
what was needed and returned a new object, but this was quite a bit slower
than using the standard lib copy.copy().
Here''s my class of the objects being copied:
class Rule(list):
def __init__(self, lhs=None, rhs=None, nClasses=0, nCases=0):
self.nClasses = nClasses
self.nCases = nCases
if lhs is not None:
self.extend(lhs)
if rhs is None:
self.rhs=tuple()
else:
self.rhs=rhs
# as I mentioned, Rule.copy is slower than copy.copy
def copy(self):
r = Rule(self,
self.rhs,
self.nClasses,
self.nCases)
return r
Basically, the left hand side of a rule (e.g., a rule is a & b & c -d) is
self and three other pieces of information are kept around, two ints and a
right hand side (e.g., (d, 5) meaning that d takes the value five ... all
the LHS elements are tuples as well).
As far as optimization goes, it seems I could write a custom __copy__
method, but a c.l.python search (sorry, forgot to bookmark the reference)
seemed to indicate that a special purpose __copy__ that copies all the
objects''s attributes would lose out to the generic copy.copy(). This
doesn''t seem quite right ... ideas?
Next option would be to rewrite the whole class in C. I haven''t done C
extensions before and since most of the class is already a builtin list,
that seems like a lot of work for little gain. Would Pyrex be a help here?
I don''t think the relevant typing information (on the ints alone, I guess)
would get me very far ... and again, recoding the list stuff (by hand or by
Pyrex) in C is not going to get any gains (it might slow things down?).
It seems to me that shallow copies (of objects built from immutable types)
should be able to be speed up by memory mapping (somehow!). The top-level
list/rule should be the only new reference that needs to be created.
Any quick answers or most likely directions to explore, would be greatly
appreciated.
Regards,
Mark
Mark E. Fenner wrote:
>
and the copy is taking the majority (42%) of my execution time.
So, I''d like to speed up my copy. I had an explicit copy method that did
what was needed and returned a new object, but this was quite a bit slower
than using the standard lib copy.copy().
How are you measuring? It seems to me that your Rule.copy method is a lot faster
than copy.copy:
>>r= Rule(range(100))
shell.timefunc(r.copy)
''copy(...) 36458 iterations, 13.71usec per call''
>>from copy import copy
shell.timefunc(copy, r)
''copy(...) 4498 iterations, 111.17usec per call''
>>>
where shell.timefunc is:
def _get_timer():
if sys.platform == "win32":
return time.clock
else:
return time.time
return
def timefunc(func, *args, **kwds):
timer = _get_timer()
count, totaltime = 0, 0
while totaltime < 0.5:
t1 = timer()
res = func(*args, **kwds)
t2 = timer()
totaltime += (t2-t1)
count += 1
if count 1000:
unit = "usec"
timeper = totaltime * 1000000 / count
else:
unit = "msec"
timeper = totaltime * 1000 / count
return "%s(...) %s iterations, %.2f%s per call" % \
(func.__name__, count, timeper, unit)
Michael
Michael Spencer wrote:
Mark E. Fenner wrote:
>>
and the copy is taking the majority (42%) of my execution time.
So, I''d like to speed up my copy. I had an explicit copy method that did
what was needed and returned a new object, but this was quite a bit
slower than using the standard lib copy.copy().
How are you measuring? It seems to me that your Rule.copy method is a lot
faster than copy.copy:
>>r= Rule(range(100))
>>shell.timefunc(r.copy)
''copy(...) 36458 iterations, 13.71usec per call''
>>from copy import copy
>>shell.timefunc(copy, r)
''copy(...) 4498 iterations, 111.17usec per call''
<snip>
MichaelMichael,
Thank you. I misinterpreted something somewhere ... the program is indeed
faster using Rule.copy. I need to look over the rest of my profiling data,
to see if I screwed up elsewhere as well.
So, how to optimize Rule.copy()?
Regards,
Mark
Mark E. Fenner wrote:
Here''s my class of the objects being copied:Here''s a couple of things that might help speed up your __init__
method, and hence your copy method:
>
class Rule(list):
def __init__(self, lhs=None, rhs=None, nClasses=0, nCases=0):def __init__(self, lhs=None, rhs=(), nClasses=0, nCases=0):
self.nClasses = nClasses
self.nCases = nCases
if lhs is not None:
self.extend(lhs)what does the extend method do? If it is small, perhaps inline a copy
of its code here.>
if rhs is None:
self.rhs=tuple()
else:
self.rhs=rhsReplace the above 4 lines by:
self.rhs = rhs
>
# as I mentioned, Rule.copy is slower than copy.copy
def copy(self):
r = Rule(self,
self.rhs,
self.nClasses,
self.nCases)
return r
HTH,
John
这篇关于优化内循环复制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
