Python线程出奇地慢 [英] Python threading unexpectedly slower

问题描述

我决定学习如何在Python中完成多线程，并且进行了比较以查看在双核CPU上可以获得什么样的性能提升.我发现我的简单多线程代码实际上比顺序运行的慢，并且我不知道为什么.

I have decided to learn how multi-threading is done in Python, and I did a comparison to see what kind of performance gain I would get on a dual-core CPU. I found that my simple multi-threaded code actually runs slower than the sequential equivalent, and I cant figure out why.

我进行的测试是生成大量随机数，然后打印出最大数量的

The test I contrived was to generate a large list of random numbers and then print the maximum

from random import random
import threading

def ox():
    print max([random() for x in xrange(20000000)])

ox()大约需要6秒才能完成我的Intel Core 2 Duo，而ox();ox()大约需要12秒.

ox() takes about 6 seconds to complete on my Intel Core 2 Duo, while ox();ox() takes about 12 seconds.

然后我尝试从两个线程调用ox()来查看完成的速度.

I then tried calling ox() from two threads to see how fast that would complete.

def go():
    r = threading.Thread(target=ox)
    r.start()
    ox()

go()大约需要18秒才能完成，两个结果在彼此之间的1秒钟之内打印出来.为什么要慢一些?

go() takes about 18 seconds to complete, with the two results printing within 1 second of eachother. Why should this be slower?

我怀疑ox()是自动并行化的，因为如果我查看Windows任务管理器的性能"选项卡，并在我的python控制台中调用ox()，则两个处理器的利用率都会提高到75％左右，直到完成为止. Python会自动并行化max()之类的东西吗?

I suspect ox() is being parallelized automatically, because I if look at the Windows task manager performance tab, and call ox() in my python console, both processors jump to about 75% utilization until it completes. Does Python automatically parallelize things like max() when it can?

推荐答案

1. Python具有GIL. Python字节码一次只能由一个处理器执行.只有某些C模块(不管理Python状态)才能同时运行.
2. Python GIL在锁定线程之间的状态方面有巨大的开销.在较新的版本或开发分支中已对此进行了修复-至少应使多线程CPU绑定代码与单线程代码一样快.

您需要使用多进程框架与Python并行化.幸运的是，Python随附的 multiprocessing 模块使这一操作变得相当容易.

You need to use a multi-process framework to parallelize with Python. Luckily, the multiprocessing module which ships with Python makes that fairly easy.

很少有语言可以自动并行化表达式.如果这是您想要的功能，我建议使用Haskell(数据并行Haskell)

Very few languages can auto-parallelize expressions. If that is the functionality you want, I suggest Haskell (Data Parallel Haskell)

这篇关于Python线程出奇地慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！