并发未来是GIL的药吗? [英] Is concurrent.futures a medicine of the GIL?

问题描述

我只是在搜索这个新的实现，我使用的是python 2.7，我必须安装此，所以如果我使用它，我会忘记CPython上的GIL吗?

I was just searching about this new implementation, and i use python 2.7, i must install this, so if i use it, i'll forget the word GIL on CPython?

推荐答案

不，concurrent.futures与GIL几乎没有任何关系.

No, concurrent.futures has almost nothing whatsoever to do with the GIL.

使用进程而不是线程是GIL的良药. (当然，与所有药物一样，它也有副作用.但是它可以起作用.)

Using processes instead of threads is medicine for the GIL. (Of course, like all medicine, it has side effects. But it works.)

与直接使用threading或multiprocessing相比，futures模块只为您提供了一种更简单的计划和等待任务的方式.而且它还具有另一个优点，即您可以在线程池和进程池(甚至可能是greenlet循环，或者您发明并构建的某些疯狂的东西)之间进行交换而无需更改future代码.因此，如果您不知道代码是否会遇到GIL问题，则可以将其构建为使用线程，然后将其切换为使用经过单行更改的进程，这非常好.

The futures module just gives you a simpler way to schedule and wait on tasks than using threading or multiprocessing directly. And it has the added advantage that you can swap between a thread pool and a process pool (and maybe even a greenlet loop, or something crazy you invent and build) without changing the future code. So, if you don't know whether your code will have GIL problems, you can build it to use threads, and then switch it to use processes with a one-line change, which is pretty nice.

但是，如果使用ThreadPoolExecutor，它将具有与使用threading和queue手动创建线程池，任务队列等完全相同的GIL问题.如果您使用ProcessPoolExecutor，它将以与手动使用multiprocessing相同的方式(并具有相同的权衡)避免GIL问题.

But, if you use the ThreadPoolExecutor, it will have the exact same GIL issues as if you created a thread pool, task queue, etc. manually with threading and queue. If you use the ProcessPoolExecutor, it will avoid the GIL issues in the same way (and with the same tradeoffs) as if you used multiprocessing manually.

PyPI软件包只是concurrent.futures模块从3.2到2.x(以及3.0-3.1)的简单反向移植. (它并没有神奇地为您提供经过改进的新的3.2 GIL，也没有为您提供经过改进的3.3 GIL，更不用说删除GIL了.)

And the PyPI package is just a simple backport of the concurrent.futures module from 3.2 to 2.x (and 3.0-3.1). (It doesn't magically give you the new-and-sort-of-improved 3.2 GIL, or the more-improved 3.3 GIL, much less remove the GIL.)

我什至不应该提到GIL的变化，因为这似乎增加了混乱……但是现在，让我尝试通过过度简化来理顺它.

I probably shouldn't even have mentioned the GIL changes, because this seems to have just added confusion… but now, let me try to straighten it out, by oversimplifying terribly.

如果除了IO受限的工作之外什么都没有，那么线程是获取并发性的一种好方法，可以达到合理的限制. 3.3确实使它们更好地工作-但是在大多数情况下，2.7已经足够好了，而在大多数情况下，2.7仍然不够好.如果要处理10000个并发客户端，则要使用事件循环(例如twisted，tornado，gevent，tulip等)而不是线程.

If you have nothing but IO-bound work, threads are a great way to get concurrency, up to a reasonable limit. And 3.3 does make them work even better—but for most cases, 2.7 is already good enough, and, for most cases where it isn't, 3.3 still isn't good enough. If you want to handle 10000 simultaneous clients, you're going to want to use an event loop (e.g., twisted, tornado, gevent, tulip, etc.) instead of threads.

如果您有任何CPU限制的工作，线程根本无法帮助并行化该工作.实际上，它们使情况变得更糟. 3.3使该惩罚不那么糟糕，但这仍然是一种惩罚，您仍然不应该这样做.如果要并行化CPU工作，则必须使用进程，而不是线程. 3.3的唯一优点是futures比multiprocessing易于使用，并且是内置的，而不需要安装它.

If you have any CPU-bound work, threads don't help parallelize that work at all. In fact, they make things worse. 3.3 makes that penalty not quite as bad, but it's still a penalty, and you still shouldn't ever do this. If you want to parallelize CPU work, you have to use processes, not threads. The only advantage of 3.3 is that futures is a little easier to use than multiprocessing, and comes built-in instead of needing to install it.

我不想阻止您升级到3.3，因为它比2.7更好地实现了更好的语言.但是更好的并发并不是迁移的理由.

I don't want to discourage you from moving to 3.3, because it's a better implementation of a better language than 2.7. But better concurrency is not a reason to move.

这篇关于并发未来是GIL的药吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！