分别重定向和重新组合stderr/stdout，而不会丢失顺序 [英] Separately redirecting and recombining stderr/stdout without losing ordering

问题描述

我想执行一条命令，并想重定向stderr和stdout，如下所示:

I want to execute a command and want to redirect stderr and stdout as below:

stderr和stdout->仅应在保持顺序的同时写入logs.log文件

stderr and stdout -> should be written ONLY to logs.log file while keeping the order

stderr->应该被打印到SCREEN并被写入error.log

stderr -> should be printed to SCREEN and also written to errors.log

到目前为止，我可以将它们重定向到屏幕和文件log.txt，如下所示:

So far I can redirect them both to screen and the file log.txt like this:

command 2>&1 | tee logs.log

但是以上不是我所需要的.

But the above is not what I need.

再次清楚表明结果是什么.

To make it more clear again what the results needs to be.

执行命令后，我只需要在屏幕上看到stderr的结果，我需要使用stderr创建一个名为errors.log的文件，而我需要使用stdout和结果同时使用另一个名为logs.log的文件.按创建它们的原始顺序显示stderr.

After command is executed I need to see on screen only the result of stderr, I need to have one file named errors.log with stderr, and I need to have another file named logs.log with the result of both stdout and stderr in the original order in which they were created.

推荐答案

从理论上讲，如果没有一些难看的黑客手段，则在执行单独的重定向时保持完美的顺序是不可能的.顺序仅保留在写操作中(在O_APPEND中)模式)直接复制到同一文件；一旦在一个进程中放入了类似tee的内容，而没有在另一个进程中放入，那么保证顺序就超出了窗口，并且如果不保留有关以什么顺序调用了哪些系统调用的信息，就无法进行检索.

Preserving perfect order while performing separate redirections is not even theoretically possible without some ugly hackery. Ordering is only preserved in writes (in O_APPEND mode) directly to the same file; as soon as you put something like tee in one process but not the other, ordering guarantees go out the window and can't be retrieved without keeping information about which syscalls were invoked in what order.

那么，那骇客会是什么样子?它可能看起来像这样:

So, what would that hackery look like? It might look something like this:

# eat our initialization time *before* we start the background process
sudo sysdig-probe-loader

# now, start monitoring syscalls made by children of this shell that write to fd 1 or 2
# ...funnel content into our logs.log file
sudo sysdig -s 32768 -b -p '%evt.buffer' \
  "proc.apid=$$ and evt.type=write and (fd.num=1 or fd.num=2)" \
  > >(base64 -i -d >logs.log) \
  & sysdig_pid=$!

# Run your-program, with stderr going both to console and to errors.log
./your-program >/dev/null 2> >(tee errors.log)

也就是说，这仍然是丑陋的黑客手段:它仅捕获直接写入FD 1和FD 2的内容，并且不跟踪可能发生的任何进一步的重定向. (可以通过执行对FIFO的写操作，并使用sysdig跟踪对这些FIFO的写操作来改善这一点；这种方式fdup()和类似的操作可以按预期工作；但是上述足以证明这一概念.)

That said, this remains ugly hackery: It only catches writes direct to FDs 1 and 2, and doesn't track any further redirections that may take place. (This could be improved by performing the writes to FIFOs, and using sysdig to track writes to those FIFOs; that way fdup() and similar operations would work as-expected; but the above suffices to prove the concept).

在这里，我们演示如何使用此方法仅对stderr进行着色，而让stdout保持不变-通过告诉sysdig生成JSON流作为输出，然后对其进行迭代:

Here we demonstrate how to use this to colorize only stderr, and leave stdout alone -- by telling sysdig to generate a stream of JSON as output, and then iterating over that:

exec {colorizer_fd}> >(
  jq --unbuffered --arg startColor "$(tput setaf 1)" --arg endColor "$(tput sgr0)" -r '
    if .["fd.filename"] == "stdout" then
      ("STDOUT: " + .["evt.buffer"])
    else
      ("STDERR: " + $startColor + .["evt.buffer"] + $endColor)
    end
  '
)

sudo sysdig -s 32768 -j -p '%fd.filename %evt.buffer' \
  "proc.apid=$$ and evt.type=write and proc.name != jq and (fd.num=1 or fd.num=2)" \
  >&$colorizer_fd \
  & sysdig_pid=$!

# Run your-program, with stdout and stderr going to two separately-named destinations
./your-program >stdout 2>stderr

因为我们要键入输出文件名(stdout和stderr)，所以对于上面的代码来说，这些文件名必须是恒定的-可以使用所需的任何临时目录.

Because we're keying off the output filenames (stdout and stderr), these need to be constant for the above code to work -- any temporary directory desired can be used.

显然，您实际上不应执行任何操作.更新程序以支持其本国语言(Java中的Log4j，Python日志记录模块等)可用的任何日志记录基础结构要明确配置其日志记录.

Obviously, you shouldn't actually do any of this. Update your program to support whatever logging infrastructure is available in its native language (Log4j in Java, the Python logging module, etc) to allow its logging to be configured explicitly.

这篇关于分别重定向和重新组合stderr/stdout，而不会丢失顺序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！