python中的多线程:大部分时间是否真的有效? [英] multi-threading in python: is it really performance effiicient most of the time?

问题描述

据我所知，在大多数情况下(不是全部)，是性能因素驱动multi-threading的编程. (与Java或Python无关).

In my little understanding, it is the performance factor that drives programming for multi-threading in most cases but not all. (irrespective of Java or Python).

我正在SO中的GIL上阅读这篇启发性文章.本文总结了python采用GIL机制；即在任何给定时间只有single Thread可以执行python byte code. 这使single thread应用程序真正更快.

I was reading this enlightening article on GIL in SO. The article summarizes that python adopts GIL mechanism; i.e only a single Thread can execute python byte code at any given time. This makes single thread application really faster.

我的问题如下:

由于如果在给定点仅服务一个Thread，那么multiprocessing或thread模块是否提供了一种克服GIL施加的限制的方法?如果没有，他们提供什么功能来进行真正的multi-task工作

Since if only one Thread is served at a given point, does multiprocessing or thread module provides a way to overcome this limitation imposed by GIL? If not, what features does they provide for doing a real multi-task work

在上述帖子的评论部分中，已接受的答案中有一个问题要问，但是还没有答案?我也有这个问题

There was a question asked in the comments section of the above post in the accepted answer,but no answer has been made? I had this question in my mind too

^so at any time point of time, only one thread will be serving content to client... 
so no point of actually using multithreading to improve performance. right?

推荐答案

您对GIL的看法是正确的，没有必要使用多线程进行CPU限制的计算，因为CPU仅由一个线程使用.

You're right about the GIL, there is no point to use multithreading to do CPU-bound computation, as the CPU will only be used by one thread.

但是前面的语句可能启发了您:如果您的计算不受CPU限制，则可以利用多线程.

But that previous statement may have enlighted you: If your computation is not CPU bound, you may take advantage of multithreading.

一个典型的例子是您的应用程序大部分时间都在等待某事.

A typical example is when your application take most of its time waiting for something.

许多非CPU绑定程序示例之一: 假设您要构建Web搜寻器，则必须搜寻许多网站并将它们存储在数据库中，这需要花费多少时间?等待服务器发送数据，实际下载数据并将其存储在数据库中，这里没有任何CPU绑定.在这里，您可能会使用一组搜寻器而不是一个单独的搜寻器来获得更快的搜寻器.通常情况下，如果一个网站几乎瘫痪且响应速度非常慢(约30秒)，在此期间，单线程应用程序将等待该网站，您将陷入困境.在多线程应用程序中，其他线程将继续爬网，这很酷.

One of many many examples of not-CPU bound program: Say you want to build a web crawler, you have to crawl many many websites, and store them in a database, what does cost times ? Waiting for the servers to send data, actually downloading the data, and storing it in the database, nothing CPU bound here. Here you may get a faster crawler using a pool of crawlers instead of one single crawler. Typically in the case one website is almost down and very slow to respond (~30s), during this time, a single-threaded application will wait for the website, you're stuck. In a multithreaded application, other threads will continue crawling, and that's cool.

另一方面，由于每个进程有一个GIL，因此您可以使用多重处理来进行CPU限制的计算.

On the other hand, as there is one GIL per process, you may use multiprocessing to do CPU-bound computation.

作为一个旁注，它存在一些不带GIL的Python或多或少的部分实现，我想提到一个我认为以一种很酷的方式实现一些很棒的东西的方法:pypy STM .您将轻松找到，搜索摆脱GIL"有关该主题的很多线索.

As a side note, it exists some more or less partial implementations of Python without the GIL, I'd like to mention one that I think is in a great way to achieve something cool: pypy STM. You'll easily find, searching "get rid of the GIL" a lot of threads about the subject.

这篇关于python中的多线程:大部分时间是否真的有效?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！