速度测试导致奇怪的行为.在一个实例中将花费的时间乘以 100，在另一个实例中仅乘以 10 [英] Speed Test causing weird behavior. Multiplying time spent by 100 in one instance, only 10 in another
问题描述
我正在使用三个函数进行速度测试，readFile、prepDict 和 test.测试只是 prepDict(readFile).然后我用 timeit 模块运行了很多次.
I am doing a speed test with three functions, readFile, prepDict and test. Test is simply prepDict(readFile). I am then running these many times with the timeit module.
当我将循环次数增加 10 倍时，函数 prepDict 需要大约 100 倍的时间，但是使用函数 prepDict 的函数 test 只增加了 10.
When I increase the number of loops by a factor of 10, function prepDict takes ~100 times longer, however function test which uses function prepDict only increases by 10.
这里是函数和测试.
def readFile(filepath):
tempDict = {}
file = open(filepath,'rb')
for line in file:
split = line.split('\t')
tempDict[split[1]] = split[2]
return tempDict
def prepDict(tempDict):
for key in tempDict.keys():
tempDict[key+'a'] = tempDict[key].upper()
del tempDict[key]
return tempDict
def test():
prepDict(readFile('two.txt'))
if __name__=='__main__':
from timeit import Timer
t = Timer(lambda: readFile('two.txt'))
print 'readFile(10000): ' + str(t.timeit(number=10000))
tempDict = readFile('two.txt')
t = Timer(lambda: prepDict(tempDict))
print 'prepDict (10000): ' + str(t.timeit(number=10000))
t = Timer(lambda: test())
print 'prepDict(readFile) (10000): ' + str(t.timeit(number=10000))
t = Timer(lambda: readFile('two.txt'))
print 'readFile(100000): ' + str(t.timeit(number=100000))
tempDict = readFile('two.txt')
t = Timer(lambda: prepDict(tempDict))
print 'prepDict (100000): ' + str(t.timeit(number=100000))
t = Timer(lambda: test())
print 'prepDict(readFile) (100000): ' + str(t.timeit(number=100000))
我得到的结果如下:
readFile(10000): 0.61602914474
prepDict (10000): 0.200615847469
prepDict(readFile) (10000): 0.609288647286
readFile(100000): 5.91858320729
prepDict (100000): 18.8842101717
prepDict(readFile) (100000): 6.45040039665
如果我多次运行它，我会得到类似的结果.为什么 prepDict 增加了约 100 倍，而 prepDict(readFile) 只增加了 10 倍，即使它使用了 prepDict 函数?
And I get similar results if I run it many times. Why does prepDict increases by a factor of ~100, while prepDict(readFile) only increases by a factor of 10, even though it is using the prepDict function?
two.txt 是一个带有这些数据点的表格分隔文件:
two.txt is a tabular delimited file with these data points:
Item Title Hello2
Item Desc Testing1232
Item Release 2011-02-03
推荐答案
这里的问题是您的 prepDict 函数扩展了输入.每次按顺序调用它时，它都有更多的数据要处理.并且该数据呈线性增长，因此第 10000 次运行所需的时间大约是第一次的 10000 倍.*
The problem here is that your prepDict function expands the inputs. Each time you call it in sequence, it has more data to deal with. And that data grows linearly, so the 10000th run takes about 10000x as long as the first.*
当你调用 test 时，它每次都会创建一个新的 dict，所以时间是恒定的.
When you call test, it's creating a new dict each time, so the time is constant.
您可以通过更改 prepDict 测试以每次在 dict 的新副本上运行来很容易地看到这一点:
You can see this pretty easily by changing the prepDict tests to run on a new copy of the dict each time:
t = Timer(lambda: prepDict(tempDict.copy()))
<小时>
顺便说一下，您的 prepDict 实际上并没有随 number 呈指数增长**，只是呈二次方增长.一般而言，当某事物呈超线性增长时，并且您想要估算算法成本，您确实需要获得两个以上的数据点.
By the way, your prepDict is not actually growing exponentially** with number, just quadratically. In general, when something is growing super-linearly, and you want to estimate the algorithmic cost, you really need to get more than two data points.
* 这不是完全 - 一旦字符串和散列操作(线性增长)所花费的时间开始淹没所有其他操作(它们是都是常数).
* That's not quite true—it only starts to grow linearly once the time taken for the string and hashing operations (which grow linearly) starts to swamp the time taken for every other operation (which are all constant).
** 你在这里没有提到指数增长，但在 您之前的问题您已经回答了，因此您可能在实际问题中做出了同样无根据的假设.
** You didn't mention anything about exponential growth here, but in your previous question you did, so you may have made the same unwarranted assumption in your real problem.
这篇关于速度测试导致奇怪的行为.在一个实例中将花费的时间乘以 100，在另一个实例中仅乘以 10的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
