python multiprocessing和内核数 [英] python multiprocessing and number of cores

问题描述

我有一个mac，它有2个物理核心和4个逻辑核心.我试图找出python multiprocessing. 据我所知，处理中的每个进程都使用计算机中的一个核心，因此我相信python脚本中的进程数不能超过计算机的核心数.这样对吗 ?

I have a mac , and it has 2 physical cores and 4 logical cores. I am trying to figure out the python multiprocessing . As far as I know, each process in the processing uses one core in the computer, so I believe the number of processes in a python script can not exceed the number of the core of computer. Is this correct ?

但是，如果我执行以下代码(它具有8个功能)，它将与具有4个功能的版本同时运行.

But if I execute the following code (it has 8 functions ) , it runs same time with the version that has 4 functions.

from multiprocessing import Process
import time 
def func1():
  for i in range(0,500):
    print "11111"
    time.sleep(0.1)


def func2():
  for i in range(500,1000):
    print "22222"
    time.sleep(0.1)

def func3():
  for i in range(1000,1500):
    print "33333"
    time.sleep(0.1)


def func4():
   for i in range(1500,2000):
    print "444444"
    time.sleep(0.1)

def func5():
   for i in range(2000,2500):
    print "555555"
    time.sleep(0.1)


def func6():
   for i in range(2500,3000):
    print "666666666"
    time.sleep(0.1)


def func7():
  for i in range(3500,4000):
    print "7777777777"
    time.sleep(0.1)

def func8():
  for i in range(4500,5000):
    print "8888888888 "
    time.sleep(0.1)

if __name__=='__main__':
  start_time = time.time()
  p1 = Process(target = func1)
  p1.start()
  p2 = Process(target = func2)
  p2.start()
  p3 = Process(target = func3)
  p3.start()
  p4 = Process(target = func4)
  p4.start()
  p5 = Process(target = func5)
  p5.start()
  p6 = Process(target = func6)
  p6.start()
  p7 = Process(target = func7)
  p7.start()
  p8 = Process(target = func8)
  p8.start()
  p1.join()
  p2.join()
  p3.join()
  p4.join()
  p5.join()
  p6.join()
  p7.join()
  p8.join()
  print "done"
  print("--- %s seconds ---" % (time.time() - start_time))

如果我用4个函数或8个函数运行此代码，执行时间是相同的，但是内核数是4，所以我认为8个函数的版本应该花费更长的时间，但事实并非如此.我在这里想念什么吗?

If I run this code with 4 functions or 8 functions, the execution time is same, but the number of cores is 4, so I think the 8 functions version should take longer, but it did not. I am missing something here ?

谢谢！

推荐答案

您可以根据需要从Python脚本启动尽可能多的进程，操作系统调度程序会将它们调度为在您的CPU内核上运行.如果这些进程受CPU限制，那么启动比内核更多的进程将不会获得更好的性能.但是在您的示例中，进程大部分时间都在睡眠，因此它们没有争夺内核，这就是为什么8个进程可以与4个同时运行的原因.

You can start as many processes from your Python script as you like and the operating system scheduler will schedule them to run on your CPU cores. If those processes are CPU bound, then you won't get better performance by starting more processes than cores. But in your example, the processes are spending most of their time sleeping, and therefore they aren't competing for the cores, which is why 8 processes can run in the same time as 4.

这篇关于python multiprocessing和内核数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！