流命令输出进度 [英] Streaming commands output progress

问题描述

我正在编写一个服务，该服务必须将已执行命令的输出流式传输到父级和日志.当有一个很长的过程时，问题是 cmd.StdoutPipe 给了我一个最终的(字符串)结果.

I'm writing a service that has to stream output of a executed command both to parent and to log. When there is a long process, the problem is that cmd.StdoutPipe gives me a final (string) result.

是否有可能给出正在发生的事情的部分输出，就像在 shell 中一样

Is it possible to give partial output of what is going on, like in shell

func main() {
    cmd := exec.Command("sh", "-c", "some long runnig task")

    stdout, _ := cmd.StdoutPipe()
    cmd.Start()

    scanner := bufio.NewScanner(stdout)
    for scanner.Scan() {
        m := scanner.Text()
        fmt.Println(m)
        log.Printf(m)
    }

    cmd.Wait()
}

附言只是输出将是:

cmd.Stdout = os.Stdout

但对我来说这还不够.

推荐答案

您发布的代码有效(执行了合理的命令).

The code you posted works (with a reasonable command executed).

这是一个用 Go 编写的简单一些长时间运行的任务"，供您调用和测试您的代码:

Here is a simple "some long running task" written in Go for you to call and test your code:

func main() {
    fmt.Println("Child started.")
    time.Sleep(time.Second*2)
    fmt.Println("Tick...")
    time.Sleep(time.Second*2)
    fmt.Println("Child ended.")
}

编译它并将其作为您的命令调用.您将看到不同的行立即出现，正如子进程所写的那样，流".

Compile it and call it as your command. You will see the different lines appear immediately as written by the child process, "streamed".

由 Scanner://golang.org/pkg/bufio/#NewScanner" rel="noreferrer">bufio.NewScanner() 读取整行并且仅在遇到换行符时返回一些内容(如bufio.ScanLines() 函数所定义).

The Scanner returned by bufio.NewScanner() reads whole lines and only returns something if a newline character is encountered (as defined by the bufio.ScanLines() function).

如果您执行的命令不打印换行符，则不会立即返回其输出(仅当打印换行符、内部缓冲区已满或进程结束时).

If the command you execute doesn't print newline characters, its output won't be returned immediately (only when newline character is printed, internal buffer is filled or the process ends).

如果您不能保证子进程打印换行符，但您仍想流式传输输出，则无法读取整行.一种解决方案是按字阅读，甚至按字符(符文)阅读.您可以通过使用 Scanner.Split() 方法:

If you have no guarantee that the child process prints newline characters but you still want to stream the output, you can't read whole lines. One solution is to read by words, or even read by characters (runes). You can achieve this by setting a different split function using the Scanner.Split() method:

scanner := bufio.NewScanner(stdout)
scanner.Split(bufio.ScanRunes)

bufio.ScanRunes 函数通过 bufio.ScanRunes 读取输入code>runes so Scanner.Scan() 将在新的 rune 可用时返回.

The bufio.ScanRunes function reads the input by runes so Scanner.Scan() will return whenever a new rune is available.

或者在没有Scanner的情况下手动阅读(在这个例子中是逐字节的):

Or reading manually without a Scanner (in this example byte-by-byte):

oneByte := make([]byte, 1)
for {
    _, err := stdout.Read(oneByte)
    if err != nil {
        break
    }
    fmt.Printf("%c", oneByte[0])
}

请注意，上面的代码会错误地读取rune UTF-8 编码中的多个字节.要读取多个 UTF-8 字节的符文，我们需要一个更大的缓冲区:

Note that the above code would read runes that multiple bytes in UTF-8 encoding incorrectly. To read multi UTF-8-byte runes, we need a bigger buffer:

oneRune := make([]byte, utf8.UTFMax)
for {
    count, err := stdout.Read(oneRune)
    if err != nil {
        break
    }
    fmt.Printf("%s", oneRune[:count])
}

注意事项

进程具有用于标准输出和标准错误(通常为几 KB 的大小)的默认缓冲区.如果进程写入标准输出或标准错误，它会进入相应的缓冲区.如果此缓冲区已满，则进一步写入将阻塞(在子进程中).如果你不读取子进程的标准输出和标准错误，你的子进程可能会在缓冲区满时挂起.

Things to keep in mind

Processes have default buffers for standard output and for standard error (usually the size of a few KB). If a process writes to the standard output or standard error, it goes into the respective buffer. If this buffer gets full, further writes will block (in the child process). If you don't read the standard output and standard error of a child process, your child process may hang if the buffer is full.

所以建议始终读取子进程的标准输出和错误.即使您知道该命令通常不会写入其标准错误，但如果发生某些错误，它可能会开始将错误消息转储到其标准错误中.

So it is recommended to always read both the standard output and error of a child process. Even if you know that the command don't normally write to its standard error, if some error occurs, it will probably start dumping error messages to its standard error.

正如 Dave C 提到的，默认情况下，子进程的标准输出和错误流被丢弃，如果不读取不会导致阻塞/挂起.但是，如果不读取错误流，您可能会错过过程中的一两件事.

As Dave C mentions by default the standard output and error streams of the child process are discarded and will not cause a block / hang if not read. But still, by not reading the error stream you might miss a thing or two from the process.

这篇关于流命令输出进度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！