Python太多的打开文件（子进程） [英] Python too many open files (subprocesses)

问题描述

当我运行一个创建大量子进程的脚本时，我似乎对Python有问题。子流程创建代码看起来类似于：

代码：

  def execute（cmd，stdout = None，stderr = subprocess.STDOUT，cwd = None）
 proc = subprocess.Popen（cmd，shell = True，stdout = stdout，stderr = stderr，cwd = cwd）
 atexit.register（lambda：__kill_proc（proc））
 return proc 
  

错误我收到的消息是：

OSError：[Errno 24]打开文件太多

一旦出现此错误，我无法创建任何进一步的子进程，直到杀死脚本并重新启动它。我想知道以下行是否可以负责。

  atexit.register（lambda：__kill_proc（proc））
  

可能是这一行创建了对子进程的引用，导致文件保持打开，直到脚本退出？

解决方案

所以似乎这行：

  atexit.register（lambda：__kill_proc（proc））
  

罪魁祸首这可能是因为Popen引用被保留，所以进程资源不是free'd。当我删除该行时，错误消失了。我现在更改了@Bakuriu建议的代码，并使用进程的pid值而不是Popen实例。

I seem to be having an issue with Python when I run a script that creates a large number of sub processes. The sub process creation code looks similar to:

Code:

def execute(cmd, stdout=None, stderr=subprocess.STDOUT, cwd=None):
    proc = subprocess.Popen(cmd, shell=True, stdout=stdout, stderr=stderr, cwd=cwd)
    atexit.register(lambda: __kill_proc(proc))
    return proc

The error message I am receiving is:

OSError: [Errno 24] Too many open files

Once this error occurs, I am unable to create any further sub processes until kill the script and start it again. I am wondering if the following line could be responsible.

atexit.register(lambda: __kill_proc(proc))

Could it be that this line creates a reference to the sub process, resulting in a "file" remaining open until the script exits?

解决方案

So it seems that the line:

atexit.register(lambda: __kill_proc(proc))

was indeed the culprit. This is probably because of the Popen reference being kept around so the process resources aren't free'd. When I removed that line the error went away. I have now changed the code as @Bakuriu suggested and am using the process' pid value rather than the Popen instance.

这篇关于Python太多的打开文件（子进程）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！