Python线程全部在单个内核上执行 [英] Python threads all executing on a single core
问题描述
我有一个Python程序，该程序产生许多线程，一次运行4个，每个线程执行一个昂贵的操作.伪代码:
I have a Python program that spawns many threads, runs 4 at a time, and each performs an expensive operation. Pseudocode:
for object in list:
t = Thread(target=process, args=(object))
# if fewer than 4 threads are currently running, t.start(). Otherwise, add t to queue
但是，当程序运行时，OS X中的活动监视器显示4个逻辑内核中的1个处于100％的状态，而其他逻辑内核中的接近0个.显然，我不能强迫OS做任何事情，但是我已经以前从来不需要像这样在多线程代码中关注性能，所以我想知道我是否只是缺少或误解了某些东西.
But when the program is run, Activity Monitor in OS X shows that 1 of the 4 logical cores is at 100% and the others are at nearly 0. Obviously I can't force the OS to do anything but I've never had to pay attention to performance in multi-threaded code like this before so I was wondering if I'm just missing or misunderstanding something.
谢谢.
推荐答案
请注意，在许多情况下(实际上，在所有情况下，昂贵的操作"都是用Python实现的计算)，由于以下原因，多个线程实际上不会并发运行: Python的全局解释器锁定(GIL).
Note that in many cases (and virtually all cases where your "expensive operation" is a calculation implemented in Python), multiple threads will not actually run concurrently due to Python's Global Interpreter Lock (GIL).
GIL是解释程序级别的锁. 此锁可防止执行 Python中一次有多个线程 口译员.每个想要的线程 运行必须等待GIL被 由另一个线程释放， 表示您的多线程Python 应用程序本质上是单一的 穿线的，对吗?是的.不完全是.
The GIL is an interpreter-level lock. This lock prevents execution of multiple threads at once in the Python interpreter. Each thread that wants to run must wait for the GIL to be released by the other thread, which means your multi-threaded Python application is essentially single threaded, right? Yes. Not exactly. Sort of.
CPython使用所谓的操作 系统"线程在幕后， 也就是说每次请求 制作一个新线程， 解释器实际上会调用 操作系统的库和 内核生成一个新线程.这 例如，与Java相同.所以 在内存中，您确实有多个 线程和正常运行 系统控制哪个线程是 预定运行.在多个 处理器机器，这意味着您 可能有很多线程分布 多个处理器，都快乐 放弃工作.
CPython uses what’s called "operating system" threads under the covers, which is to say each time a request to make a new thread is made, the interpreter actually calls into the operating system’s libraries and kernel to generate a new thread. This is the same as Java, for example. So in memory you really do have multiple threads and normally the operating system controls which thread is scheduled to run. On a multiple processor machine, this means you could have many threads spread across multiple processors, all happily chugging away doing work.
但是，尽管CPython确实使用了 操作系统线程(理论上 允许多个线程执行 在口译员内 同时)，口译员也 强制GIL被 线程才能访问 解释器和堆栈，可以修改 所有内存中的Python对象 威利·尼利.后一点是为什么 GIL存在:GIL可以防止 同时访问Python对象 通过多个线程.但这不是 救救您(如银行所说明 例如)从锁敏感 生物你不会搭便车. GIL在那里保护 口译员的记忆，而不是你的理智.
However, while CPython does use operating system threads (in theory allowing multiple threads to execute within the interpreter simultaneously), the interpreter also forces the GIL to be acquired by a thread before it can access the interpreter and stack and can modify Python objects in memory all willy-nilly. The latter point is why the GIL exists: The GIL prevents simultaneous access to Python objects by multiple threads. But this does not save you (as illustrated by the Bank example) from being a lock-sensitive creature; you don’t get a free ride. The GIL is there to protect the interpreters memory, not your sanity.
See the Global Interpreter Lock section of Jesse Noller's post for more details.
要解决此问题，请查看 Python的多处理模块.
To get around this problem, check out Python's multiprocessing module.
多个进程(明智使用) IPC)[]更好 为多CPU编写应用程序的方法 盒子而不是线程.
multiple processes (with judicious use of IPC) are[...] a much better approach to writing apps for multi-CPU boxes than threads.
- Guido van Rossum (Python的创建者)
这篇关于Python线程全部在单个内核上执行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
