子进程的Popen打开文件过多错误 [英] Too many open files error with Popen of subprocess

问题描述

我正在使用Python的子进程模块来调用命令以将值从文件写入内存.看起来像:

I'm using Python's subprocess module to call a command to write values from a file to memory. It looks like:

import subprocess

f = open('memdump', 'r')
content = [line.split()[1] for line in f]
f.close()
tbl_pt0 = 0x4400
tbl_pt1 = 0x4800
process = 0
i = 0
for value in content:
    p1 = subprocess.Popen("echo \"jaguar instance=0; jaguar wr offset=0x%x value=%s\" | pdt" \
% (tbl_pt0, value), shell = True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True)
    p2 = subprocess.Popen("echo \"jaguar instance=0; jaguar wr offset=0x%x value=%s\" | pdt" \
% (tbl_pt1, value), shell = True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True)
    tbl_pt0 += 0x4
    tbl_pt1 += 0x4

此处pdt是一个写入值的程序.该文件中有256个值，因此for循环运行256个循环.但是当我进入253循环时，subprocess.py中出现太多打开文件错误.有人有主意吗?谢谢.

Here the pdt is a program that writes the value. There're 256 values in the file, so the for loop runs 256 loops. But when I get to loop 253, I got too many open files error in subprocess.py. Any one has any idea? Thanks.

推荐答案

原因是 subprocess.Popen 调用是异步的，因为它立即返回而无需等待生成的进程退出.也就是说，您正在快速创建2x256进程.它们每个都有2个管道，每个管道占用1个文件描述符.单个进程可以一次打开有限数量的文件描述符，而您到达它是因为您不必等待进程和管道关闭.您可以等待他们退出，例如 p.communicate()，其中 p 是 subprocess.Popen 的返回值，或增加一次打开的最大文件描述符:

The reason is that subprocess.Popen call is asynchronous in that it returns immediately without waiting for the spawned process to exit. That said, you are quickly creating 2x256 processes. Each of them has 2 pipes each taking up 1 file descriptor. A single process can have a limited amount of file descriptors opened at any one time, and you are reaching it because you don't wait for the processes and pipes to close. You can either wait for them to exit, e.g. p.communicate() where p is the return value of subprocess.Popen, or increase the maximum file descriptors opened at once:

• 永久-将 fs.file-max = 100000 添加到/etc/sysctl.conf
• 暂时(直到重新启动)- sysctl -w fs.file-max = 100000

这篇关于子进程的Popen打开文件过多错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！