在 Ruby 中从外部进程的 STDOUT 连续读取 [英] Continuously read from STDOUT of external process in Ruby

问题描述

我想通过 ruby​​ 脚本从命令行运行 Blender，然后该脚本将逐行处理 Blender 给出的输出以更新 GUI 中的进度条.Blender 是我需要读取其标准输出的外部进程并不重要.

I want to run blender from the command line through a ruby script, which will then process the output given by blender line by line to update a progress bar in a GUI. It's not really important that blender is the external process whose stdout I need to read.

当 Blender 进程仍在运行时，我似乎无法捕捉到 Blender 通常打印到 shell 的进度消息，我尝试了几种方法.我似乎总是在搅拌机退出后访问搅拌机的标准输出，而不是在它仍在运行时.

I can't seem to be able to catch the progress messages blender normally prints to the shell when the blender process is still running, and I've tried a few ways. I always seem to access the stdout of blender after blender has quit, not while it's still running.

这是一个失败的尝试示例.它确实获取并打印了 Blender 输出的前 25 行，但只有在 Blender 进程退出之后:

Here's an example of a failed attempt. It does get and print the first 25 lines of the output of blender, but only after the blender process has exited:

blender = nil
t = Thread.new do
  blender = open "| blender -b mball.blend -o //renders/ -F JPEG -x 1 -f 1"
end
puts "Blender is doing its job now..."
25.times { puts blender.gets}

为了更清楚一点，调用 Blender 的命令在 shell 中返回一个输出流，指示进度(第 1-16 部分已完成等).似乎任何对获取"输出的调用都被阻止，直到搅拌机退出.问题是如何在搅拌机仍在运行时访问此输出，因为搅拌机将其输出打印到外壳.

To make it a little clearer, the command invoking blender gives back a stream of output in the shell, indicating progress (part 1-16 completed etc). It seems that any call to "gets" the output is blocked until blender quits. The issue is how to get access to this output while blender is still running, as blender prints it's output to shell.

推荐答案

我在解决我的这个问题方面取得了一些成功.这是详细信息，并附有一些解释，以防遇到类似问题的人找到此页面.但如果您不关心细节，这里是简短的回答:

I've had some success in solving this problem of mine. Here are the details, with some explanations, in case anyone having a similar problem finds this page. But if you don't care for details, here's the short answer:

按以下方式使用 PTY.spawn(当然是使用您自己的命令):

Use PTY.spawn in the following manner (with your own command of course):

require 'pty'
cmd = "blender -b mball.blend -o //renders/ -F JPEG -x 1 -f 1" 
begin
  PTY.spawn( cmd ) do |stdout, stdin, pid|
    begin
      # Do stuff with the output here. Just printing to show it works
      stdout.each { |line| print line }
    rescue Errno::EIO
      puts "Errno:EIO error, but this probably just means " +
            "that the process has finished giving output"
    end
  end
rescue PTY::ChildExited
  puts "The child process exited!"
end

而且这是一个很长的答案，有太多的细节:

真正的问题似乎是，如果一个进程没有明确地刷新它的标准输出，那么写入标准输出的任何内容都会被缓冲而不是实际发送，直到进程完成，以最小化 IO(这是 显然是许多 C 库的实现细节，目的是通过较少的 IO 来最大化吞吐量).如果您可以轻松修改该过程以使其定期刷新 stdout，那么这将是您的解决方案.就我而言，它是搅拌机，所以对于像我这样的完整菜鸟来说，修改源代码有点吓人.

The real issue seems to be that if a process doesn't explicitly flush its stdout, then anything written to stdout is buffered rather than actually sent, until the process is done, so as to minimize IO (this is apparently an implementation detail of many C libraries, made so that throughput is maximized through less frequent IO). If you can easily modify the process so that it flushes stdout regularly, then that would be your solution. In my case, it was blender, so a bit intimidating for a complete noob such as myself to modify the source.

但是当您从 shell 运行这些进程时，它们会实时向 shell 显示标准输出，并且标准输出似乎没有被缓冲.它仅在从我相信的另一个进程调用时才被缓冲，但如果正在处理 shell，则会实时看到标准输出，无缓冲.

But when you run these processes from the shell, they display stdout to the shell in real-time, and the stdout doesn't seem to be buffered. It's only buffered when called from another process I believe, but if a shell is being dealt with, the stdout is seen in real time, unbuffered.

这种行为甚至可以通过 ruby​​ 进程作为子进程观察到，其输出必须实时收集.只需使用以下行创建一个脚本 random.rb:

This behavior can even be observed with a ruby process as the child process whose output must be collected in real time. Just create a script, random.rb, with the following line:

5.times { |i| sleep( 3*rand ); puts "#{i}" }

然后是一个 ruby​​ 脚本来调用它并返回它的输出:

Then a ruby script to call it and return its output:

IO.popen( "ruby random.rb") do |random|
  random.each { |line| puts line }
end

您会发现，您并没有像预期的那样实时获得结果，而是在之后立即获得结果.STDOUT 正在被缓冲，即使您自己运行 random.rb，它也不会被缓冲.这可以通过在 random.rb 的块内添加一个 STDOUT.flush 语句来解决.但是，如果您无法更改源，则必须解决此问题.您无法从进程外部刷新它.

You'll see that you don't get the result in real-time as you might expect, but all at once afterwards. STDOUT is being buffered, even though if you run random.rb yourself, it isn't buffered. This can be solved by adding a STDOUT.flush statement inside the block in random.rb. But if you can't change the source, you have to work around this. You can't flush it from outside the process.

如果子进程可以实时打印到 shell，那么必须有一种方法可以用 Ruby 实时捕获它.有.你必须使用 PTY 模块，我相信它包含在 ruby​​ 核心中(无论如何都是 1.8.6).可悲的是它没有记录.但我幸运地找到了一些使用示例.

If the subprocess can print to shell in real-time, then there must be a way to capture this with Ruby in real-time as well. And there is. You have to use the PTY module, included in ruby core I believe (1.8.6 anyways). Sad thing is that it's not documented. But I found some examples of use fortunately.

首先，解释一下 PTY 是什么，它代表 伪终端.基本上，它允许 ruby​​ 脚本将自身呈现给子进程，就好像它是一个刚刚将命令输入到 shell 中的真实用户一样.因此，只有当用户通过 shell 启动进程时才会发生任何改变的行为(例如，在这种情况下，STDOUT 没有被缓冲).隐藏另一个进程已启动此进程的事实允许您实时收集 STDOUT，因为它没有被缓冲.

First, to explain what PTY is, it stands for pseudo terminal. Basically, it allows the ruby script to present itself to the subprocess as if it's a real user who has just typed the command into a shell. So any altered behavior that occurs only when a user has started the process through a shell (such as the STDOUT not being buffered, in this case) will occur. Concealing the fact that another process has started this process allows you to collect the STDOUT in real-time, as it isn't being buffered.

要使用 random.rb 脚本作为子进程进行此操作，请尝试以下代码:

To make this work with the random.rb script as the child, try the following code:

require 'pty'
begin
  PTY.spawn( "ruby random.rb" ) do |stdout, stdin, pid|
    begin
      stdout.each { |line| print line }
    rescue Errno::EIO
    end
  end
rescue PTY::ChildExited
  puts "The child process exited!"
end

这篇关于在 Ruby 中从外部进程的 STDOUT 连续读取的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！