Python 线程意外地变慢 [英] Python threading unexpectedly slower

问题描述

我决定学习 Python 中的多线程是如何完成的，我做了一个比较，看看我会在双核 CPU 上获得什么样的性能提升.我发现我的简单多线程代码实际上比顺序等效代码运行得慢，我不知道为什么.

I have decided to learn how multi-threading is done in Python, and I did a comparison to see what kind of performance gain I would get on a dual-core CPU. I found that my simple multi-threaded code actually runs slower than the sequential equivalent, and I cant figure out why.

我设计的测试是生成一个很大的随机数列表，然后打印最大值

The test I contrived was to generate a large list of random numbers and then print the maximum

from random import random
import threading

def ox():
    print max([random() for x in xrange(20000000)])

ox() 在我的 Intel Core 2 Duo 上大约需要 6 秒才能完成，而 ox();ox() 大约需要 12 秒.

ox() takes about 6 seconds to complete on my Intel Core 2 Duo, while ox();ox() takes about 12 seconds.

然后我尝试从两个线程调用 ox() 以查看完成的速度.

I then tried calling ox() from two threads to see how fast that would complete.

def go():
    r = threading.Thread(target=ox)
    r.start()
    ox()

go() 大约需要 18 秒才能完成，两个结果在 1 秒内打印出来.为什么这会更慢?

go() takes about 18 seconds to complete, with the two results printing within 1 second of eachother. Why should this be slower?

我怀疑 ox() 正在自动并行化，因为如果查看 Windows 任务管理器性能选项卡，并在我的 python 控制台中调用 ox()，两者处理器跳到大约 75% 的利用率，直到它完成.Python 是否会自动并行化诸如 max() 之类的东西?

I suspect ox() is being parallelized automatically, because I if look at the Windows task manager performance tab, and call ox() in my python console, both processors jump to about 75% utilization until it completes. Does Python automatically parallelize things like max() when it can?

推荐答案

1. Python 具有 GIL.Python 字节码一次只能由一个处理器执行.只有某些 C 模块(不管理 Python 状态)才能同时运行.
2. Python GIL 在锁定线程之间的状态方面有巨大的开销.在较新的版本或开发分支中对此进行了修复 - 至少应该使多线程 CPU 绑定代码与单线程代码一样快.

需要使用多进程框架与 Python 并行化.幸运的是，Python 附带的 multiprocessing 模块使这变得相当容易.

You need to use a multi-process framework to parallelize with Python. Luckily, the multiprocessing module which ships with Python makes that fairly easy.

很少有语言可以自动并行化表达式.如果这是你想要的功能，我建议使用 Haskell(Data Parallel Haskell)

Very few languages can auto-parallelize expressions. If that is the functionality you want, I suggest Haskell (Data Parallel Haskell)

这篇关于Python 线程意外地变慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！