在不丢失顺序的情况下单独重定向和重新组合 stderr/stdout [英] Separately redirecting and recombining stderr/stdout without losing ordering

问题描述

我想执行一个命令并想重定向 stderr 和 stdout 如下:

I want to execute a command and want to redirect stderr and stdout as below:

stderr 和 stdout -> 应该只写入 logs.log 文件，同时保持顺序

stderr and stdout -> should be written ONLY to logs.log file while keeping the order

stderr -> 应该打印到 SCREEN 并写入 errors.log

stderr -> should be printed to SCREEN and also written to errors.log

到目前为止，我可以像这样将它们重定向到屏幕和文件 log.txt:

So far I can redirect them both to screen and the file log.txt like this:

command 2>&1 | tee logs.log

但以上不是我需要的.

再次明确需要什么结果.

To make it more clear again what the results needs to be.

命令执行后我只需要在屏幕上看到stderr的结果，我需要一个名为errors.log的文件和stderr，我需要另一个名为logs.log的文件，其中包含stdout和stderr的结果stderr 按照创建时的原始顺序.

After command is executed I need to see on screen only the result of stderr, I need to have one file named errors.log with stderr, and I need to have another file named logs.log with the result of both stdout and stderr in the original order in which they were created.

推荐答案

在执行单独重定向的同时保持完美的顺序在理论上甚至在没有一些丑陋的技巧的情况下是不可能的. 排序只保留在写入中(在 O_APPEND 中)模式)直接到同一个文件；一旦您将诸如 tee 之类的东西放在一个进程中而不是另一个进程中，排序保证就会消失，并且无法在不保留有关以何种顺序调用哪些系统调用的信息的情况下进行检索.

Preserving perfect order while performing separate redirections is not even theoretically possible without some ugly hackery. Ordering is only preserved in writes (in O_APPEND mode) directly to the same file; as soon as you put something like tee in one process but not the other, ordering guarantees go out the window and can't be retrieved without keeping information about which syscalls were invoked in what order.

那么，那个黑客会是什么样子?它可能看起来像这样:

So, what would that hackery look like? It might look something like this:

# eat our initialization time *before* we start the background process
sudo sysdig-probe-loader

# now, start monitoring syscalls made by children of this shell that write to fd 1 or 2
# ...funnel content into our logs.log file
sudo sysdig -s 32768 -b -p '%evt.buffer' 
  "proc.apid=$$ and evt.type=write and (fd.num=1 or fd.num=2)" 
  > >(base64 -i -d >logs.log) 
  & sysdig_pid=$!

# Run your-program, with stderr going both to console and to errors.log
./your-program >/dev/null 2> >(tee errors.log)

也就是说，这仍然是丑陋的黑客:它只捕获直接写入 FD 1 和 2 的写入，并且不跟踪可能发生的任何进一步重定向.(这可以通过执行对 FIFO 的写入并使用 sysdig 跟踪对这些 FIFO 的写入来改进；这样 fdup() 和类似的操作将按预期工作；但以上足以证明概念).

That said, this remains ugly hackery: It only catches writes direct to FDs 1 and 2, and doesn't track any further redirections that may take place. (This could be improved by performing the writes to FIFOs, and using sysdig to track writes to those FIFOs; that way fdup() and similar operations would work as-expected; but the above suffices to prove the concept).

在这里，我们演示了如何使用它来仅对 stderr 着色，而将 stdout 置之不理——通过告诉 sysdig 生成 JSON 流作为输出，然后对其进行迭代:

Here we demonstrate how to use this to colorize only stderr, and leave stdout alone -- by telling sysdig to generate a stream of JSON as output, and then iterating over that:

exec {colorizer_fd}> >(
  jq --unbuffered --arg startColor "$(tput setaf 1)" --arg endColor "$(tput sgr0)" -r '
    if .["fd.filename"] == "stdout" then
      ("STDOUT: " + .["evt.buffer"])
    else
      ("STDERR: " + $startColor + .["evt.buffer"] + $endColor)
    end
  '
)

sudo sysdig -s 32768 -j -p '%fd.filename %evt.buffer' 
  "proc.apid=$$ and evt.type=write and proc.name != jq and (fd.num=1 or fd.num=2)" 
  >&$colorizer_fd 
  & sysdig_pid=$!

# Run your-program, with stdout and stderr going to two separately-named destinations
./your-program >stdout 2>stderr

因为我们正在关闭输出文件名(stdout 和 stderr)，这些需要保持不变才能使上述代码工作——任何需要的临时目录都可以使用.

Because we're keying off the output filenames (stdout and stderr), these need to be constant for the above code to work -- any temporary directory desired can be used.

显然，您实际上不应该执行任何这些操作.更新您的程序以支持其本机语言(Java 中的 Log4j、Python 日志模块等)中可用的任何日志记录基础结构，以允许其日志记录需要明确配置.

Obviously, you shouldn't actually do any of this. Update your program to support whatever logging infrastructure is available in its native language (Log4j in Java, the Python logging module, etc) to allow its logging to be configured explicitly.

这篇关于在不丢失顺序的情况下单独重定向和重新组合 stderr/stdout的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！