Python是否支持多线程?可以加快执行时间吗? [英] Does Python support multithreading? Can it speed up execution time?

问题描述

我对多线程是否适用于Python感到有些困惑.

I'm slightly confused about whether multithreading works in Python or not.

我知道对此有很多疑问，我已经阅读了许多问题，但我仍然感到困惑.我从自己的经验中知道，并且看到其他人在StackOverflow上发表了自己的答案和示例，说在Python中确实可以实现多线程.那么为什么每个人都说Python被GIL锁定并且一次只能运行一个线程呢?显然确实有效.还是我不来这里有什么区别?

I know there has been a lot of questions about this and I've read many of them, but I'm still confused. I know from my own experience and have seen others post their own answers and examples here on StackOverflow that multithreading is indeed possible in Python. So why is it that everyone keep saying that Python is locked by the GIL and that only one thread can run at a time? It clearly does work. Or is there some distinction I'm not getting here?

许多发帖者/受访者还不断提到线程是有限的，因为它不使用多个核心.但是我会说它们仍然有用，因为它们可以同时工作，因此可以更快地完成合并的工作量.我的意思是为什么还要有Python线程模块呢?

Many posters/respondents also keep mentioning that threading is limited because it does not make use of multiple cores. But I would say they are still useful because they do work simultaneously and thus get the combined workload done faster. I mean why would there even be a Python thread module otherwise?

更新:

感谢您到目前为止的所有回答.据我了解，多线程只能对某些IO任务并行运行，但一次只能运行一个CPU绑定的多个核心任务.

Thanks for all the answers so far. The way I understand it is that multithreading will only run in parallel for some IO tasks, but can only run one at a time for CPU-bound multiple core tasks.

我并不完全确定这实际上对我意味着什么，所以我仅举一个我想进行多线程的任务类型的例子.例如，假设我要遍历很长的字符串列表，并且希望对每个列表项执行一些基本的字符串操作.如果拆分列表，将每个要由我的循环/字符串代码处理的子列表发送到新线程中，然后将结果发送回队列中，这些工作负载是否会大致同时运行?最重要的是，从理论上讲，这会加快运行脚本的时间吗?

I'm not entirely sure what this means for me in practical terms, so I'll just give an example of the kind of task I'd like to multithread. For instance, let's say I want to loop through a very long list of strings and I want to do some basic string operations on each list item. If I split up the list, send each sublist to be processed by my loop/string code in a new thread, and send the results back in a queue, will these workloads run roughly at the same time? Most importantly will this theoretically speed up the time it takes to run the script?

另一个例子可能是，如果我可以在四个不同的线程中使用PIL渲染和保存四张不同的图片，并且这比一张又一张地处理图片要快吗?我想这个速度要素是我真正想知道的，而不是正确的术语.

Another example might be if I can render and save four different pictures using PIL in four different threads, and have this be faster than processing the pictures one by one after each other? I guess this speed-component is what I'm really wondering about rather than what the correct terminology is.

我也了解多处理模块，但是我现在的主要兴趣是中小型任务负载(10-30秒)，因此我认为多线程将更合适，因为子进程的启动速度可能很慢.

I also know about the multiprocessing module but my main interest right now is for small-to-medium task loads (10-30 secs) and so I think multithreading will be more appropriate because subprocesses can be slow to initiate.

推荐答案

GIL不会阻止线程化. GIL所做的全部工作就是确保一次只有一个线程在执行Python代码；控制仍在线程之间切换.

The GIL does not prevent threading. All the GIL does is make sure only one thread is executing Python code at a time; control still switches between threads.

GIL当时阻止的事情是利用多个CPU内核或单独的CPU并行运行线程.

What the GIL prevents then, is making use of more than one CPU core or separate CPUs to run threads in parallel.

这仅适用于Python代码. C扩展可以并且确实可以释放GIL，以允许C代码的多个线程和一个Python线程跨多个内核运行.这扩展到了由内核控制的I/O，例如对套接字进行读写的select()调用，使Python在多线程多核设置中合理有效地处理了网络事件.

This only applies to Python code. C extensions can and do release the GIL to allow multiple threads of C code and one Python thread to run across multiple cores. This extends to I/O controlled by the kernel, such as select() calls for socket reads and writes, making Python handle network events reasonably efficiently in a multi-threaded multi-core setup.

许多服务器部署随后要做的是运行多个Python进程，以使OS处理进程之间的调度，从而最大程度地利用您的CPU内核.您也可以使用 multiprocessing库来处理多个进程之间的并行处理一个代码库和一个父进程(如果适合您的用例).

What many server deployments then do, is run more than one Python process, to let the OS handle the scheduling between processes to utilize your CPU cores to the max. You can also use the multiprocessing library to handle parallel processing across multiple processes from one codebase and parent process, if that suits your use cases.

请注意，GIL仅适用于CPython实现； Jython和IronPython使用不同的线程实现(分别是本机Java VM和.NET通用运行时线程).

Note that the GIL is only applicable to the CPython implementation; Jython and IronPython use a different threading implementation (the native Java VM and .NET common runtime threads respectively).

直接处理更新:使用纯Python代码尝试从并行执行中提高速度的任何任务都不会看到加速，因为线程化的Python代码一次只能锁定一个线程.但是，如果混用C扩展名和I/O(例如PIL或numpy操作)，则任何C代码都可以与一个活动Python线程并行运行.

To address your update directly: Any task that tries to get a speed boost from parallel execution, using pure Python code, will not see a speed-up as threaded Python code is locked to one thread executing at a time. If you mix in C extensions and I/O, however (such as PIL or numpy operations) and any C code can run in parallel with one active Python thread.

Python线程非常适合用于创建响应式GUI或处理多个简短的Web请求，而I/O比Python代码更是瓶颈.它不适用于并行计算量大的Python代码，对于此类任务，请坚持使用multiprocessing模块或委托给专用的外部库.

Python threading is great for creating a responsive GUI, or for handling multiple short web requests where I/O is the bottleneck more than the Python code. It is not suitable for parallelizing computationally intensive Python code, stick to the multiprocessing module for such tasks or delegate to a dedicated external library.

这篇关于Python是否支持多线程?可以加快执行时间吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！