subprocess.wait()不等待Popen进程完成(使用线程时)? [英] subprocess.wait() not waiting for Popen process to finish (when using threads)?

问题描述

使用subprocess.Popen()从我的python脚本使用线程生成同一应用程序的多个实例以使其同时运行时，我遇到了一些问题.在每个线程中，我都使用popen()调用运行该应用程序，然后通过调用wait()等待它完成.问题似乎是wait()调用实际上没有等待进程完成.我尝试仅使用一个线程，并在进程开始和结束时打印出文本消息进行了实验.所以线程函数看起来像这样:

I am experiencing some problems when using subprocess.Popen() to spawn several instances of the same application from my python script using threads to have them running simultaneously. In each thread I run the application using the popen() call, and then I wait for it to finish by callingwait(). The problem seems to be that the wait()-call does not actually wait for the process to finish. I experimented by using only one thread, and by printing out text messages when the process starts, and when it finishes. So the thread function would look something like this:

def worker():
    while True:
        job = q.get() # q is a global Queue of jobs
        print('Starting process %d' % job['id'])
        proc = subprocess.Popen(job['cmd'], shell=True)
        proc.wait()
        print('Finished process %d' % job['id'])
        job.task_done()

但是，即使我只使用一个线程，它也会在出现任何已完成进程..."消息之前打印出几条正在启动进程..."消息.是否有wait()实际上没有等待的情况?我有几个不同的外部应用程序(C ++控制台应用程序)，这些应用程序又将同时运行多个实例，对于某些实例，我的代码有效，但对于其他实例，则不会.外部应用程序是否存在某些问题，以某种方式影响对wait()的调用? 创建线程的代码如下所示:

But even when I only use one thread, it will print out several "Starting process..." messages, before any "Finished process..." message appears. Are there any cases when wait() does not actually wait? I have several different external applications (C++ console applications), which in turn will have several instances running simultaneously, and for some of them my code works, but for others it won't. Could there be some issue with the external applications that somehow affects the call to wait()? The code for creating the threads looks something like this:

for i in range(1):
    t = Thread(target=worker)
    t.daemon = True
    t.start()
q.join() # Wait for the queue to empty

更新1 : 我还应该补充一点，对于某些外部应用程序，有时会得到-1073471801的返回码(proc.returncode).例如，外部应用程序之一将在前两次调用Popen时给出返回代码，而在后两次(当我有四个作业时)不给出返回代码.

Update 1: I should also add that for some of the external applications I sometimes get a return code (proc.returncode) of -1073471801. For example, one of the external applications will give that return code the first two times Popen is called, but not the last two (when I have four jobs).

更新2 : 为了解决问题，现在队列中有四个作业，这是四个不同的测试用例.当我运行代码时，对于外部应用程序之一，前两个Popen调用会生成返回代码-1073471801.但是，如果我打印出Popen调用的确切命令，并在命令窗口中运行它，则执行该命令不会出现任何问题.

Update 2: To clear things up, right now I have four jobs in the queue, which are four different test cases. When I run my code, for one of the external applications the first two Popen-calls generate the return code -1073471801. But if I print the exact command which Popen calls, and run it in a command window, it executes without any problems.

已解决！ 我设法解决了自己遇到的问题.我认为问题是我缺乏线程编程经验.我错过了这样一个事实，当我创建了我的第一个工作线程时，它们将继续存在直到python脚本退出.每次我将新项目放入队列时，我都会错误地创建更多的工作线程(对于要运行的每个外部程序，我都会成批执行).因此，当我进入第四个外部应用程序时，尽管我只以为有一个线程，但我同时运行了四个线程.

Solved! I managed to solve the issues I was having. I think the problem was my lack of experience in threaded programming. I missed the fact that when I had created my first worker threads, they would keep on living until the python script exits. By mistake I created more worker threads each time I put new items on the queue (I do that in batches for every external program I want to run). So by the time I got to the fourth external application, I had four threads running simultaneously even though I only thought I had one.

推荐答案

您还可以使用 check_call() 而不是Popen.即使在shell=True时，check_call()也要等待命令完成，然后返回作业的退出代码.

You could also use check_call() instead of Popen. check_call() waits for the command to finish, even when shell=True and then returns the exit code of the job.

这篇关于subprocess.wait()不等待Popen进程完成(使用线程时)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！