使用OMP_NUM_THREADS = 1进行Python多处理 [英] Use of OMP_NUM_THREADS=1 for Python Multiprocessing
问题描述
我听说在调用使用多处理功能的Python脚本之前使用OMP_NUM_THREADS=1可以使脚本更快.
I heard that using OMP_NUM_THREADS=1 before calling a Python script that use multiprocessing make the script faster.
是真的吗?如果是，为什么呢?
Is it true or not ? If yes, why so ?
推荐答案
Since you said in a comment that your Python program is calling a C module that uses OpenMP:
OpenMP在一个进程中执行多线程，默认的线程数通常是CPU可以同时实际运行的数量. (通常是CPU内核的数量，如果CPU具有 SMT >功能，例如Intel的Hyper-Threading.)因此，例如，如果您有四核非超线程CPU，则OpenMP将默认运行4个线程.
OpenMP does multi-threading within a process, and the default number of threads is typically the number that the CPU can actually run simultaneously. (This is generally the number of CPU cores, or a multiple of that number if the CPU has an SMT feature such as Intel's Hyper-Threading.) So if you have, for example, a quad-core non-hyperthreaded CPU, OpenMP will want to run 4 threads by default.
当您使用Python的multiprocessing模块时，您的程序将启动多个可以同时运行的Python进程.您可以控制进程数，但是通常您希望它是CPU内核/线程数，例如由multiprocessing.cpu_count()返回.
When you use Python's multiprocessing module, your program starts multiple Python processes which can run simultaneously. You can control the number of processes, but often you'll want it to be the number of CPU cores/threads, e.g. returned by multiprocessing.cpu_count().
那么，如果您运行一个运行4个Python进程的multiprocessing程序，并且每个调用一个OpenMP函数运行4个线程，那么在该四核CPU上会发生什么?您最终在4个内核上运行了16个线程.这会奏效，但不会达到最高效率，因为每个内核都必须花一些时间在任务之间进行切换.
So, what happens on that quad-core CPU if you run a multiprocessing program that runs 4 Python processes, and each calls an OpenMP function runs 4 threads? You end up running 16 threads on 4 cores. That'll work, but not at peak efficiency, since each core will have to spend some time switching between tasks.
设置OMP_NUM_THREADS=1基本上会关闭OpenMP多线程，因此您的每个Python进程都保持单线程.
Setting OMP_NUM_THREADS=1 basically turns off the OpenMP multi-threading, so each of your Python processes remains single-threaded.
但是，请确保您启动了足够的Python进程！如果您有4个CPU内核，并且仅运行2个单线程Python进程，则将使用2个内核，另外2个处于空闲状态. (在这种情况下，您可能需要设置OMP_NUM_THREADS=2.)
Make sure you're starting enough Python processes if you do this, though! If you have 4 CPU cores and you only run 2 single-threaded Python processes, you'll have 2 cores utilized and the other 2 sitting idle. (In this case you might want to set OMP_NUM_THREADS=2.)
这篇关于使用OMP_NUM_THREADS = 1进行Python多处理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
