Python多线程max_workers [英] Python multithreading max_workers
问题描述
根据的文档 ThreadPoolExecutor
According to the documentation of ThreadPoolExecutor
如果
max_workers是不提供或不给出，它将默认为计算机上的处理器数量。
If
max_workersisNoneor not given, it will default to the number of processors on the machine.
如果我没有将其设置为这样的值
If I don't set it a value like this
ThreadPoolExecutor(max_workers=None)
如果我的价值很低（2），这是否会对性能造成不利影响？
python是否已经为None值分配了所有CPU进程，而为带有数字的值仅分配了2个？
is it bad for performance in case that my value is very low (2) ? Will python already allocate all the CPU processes for None value vs allocate only 2 for value with a number?
推荐答案
首先，您似乎在链接中引用了文档中错误的部分，即用于进程而不是线程的部分。 用于 concurrent.futures.ThreadPoolExecutor 状态：
To begin with, you seem to be quoting the wrong part of the documentation in your link, namely the one for processes, not threads. The one for concurrent.futures.ThreadPoolExecutor states:
在版本3.5中进行了更改：如果
max_workers是None或未给出，它将默认为计算机上的处理器数量乘以5，假设ThreadPoolExecutor通常用于重叠I / O而不是CPU工作并且工人的数量应高于ProcessPoolExecutor的工人数量。
Changed in version 3.5: If
max_workersisNoneor not given, it will default to the number of processors on the machine, multiplied by 5, assuming that ThreadPoolExecutor is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for ProcessPoolExecutor.
由于您使用的是线程而不是进程，因此假设您的应用程序是IO绑定的，而不是CPU绑定的，并且您将其用于并发，而不是并行性。您使用的线程越多，将达到的并发程度（最高）（但要达到一个点），但是获得的CPU周期就越少（因为会有上下文切换）。您必须在典型的工作负载下对应用程序进行检测，以查看最适合您的应用程序。没有通用的最佳解决方案。
Since you're using threads, not processes, the assumption is that your application is IO bound, not CPU bound, and that you're using this for concurrency, not parallelism. The more threads you use, the higher concurrency you'll achieve (up to a point), but the less CPU cycles you'll get (as there will be context switches). You have to instrument your application under typical workloads to see what works best for you. There is no universally optimal solution for this.
这篇关于Python多线程max_workers的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
