将tcpdump输出写入压缩/ gzip压缩文件 [英] Write tcpdump output to compressed / gziped file

问题描述

我想将 tcpdump 的文本输出写入压缩文件。

I want to write the textual output of tcpdump to a compressed file.

首先，我尝试了最多显而易见：

First I tried the most obvious:

# tcpdump -l -i eth0 | gzip -c > test.gz
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C63 packets captured
244 packets received by filter
0 packets dropped by kernel
4 packets dropped by interface

# file test.gz
test.gz: empty
# 

然后我找到了 Debian 9（拉伸）：

# tcpdump -l -i eth0 | ( gzip -c > test.gz & )
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C150 packets captured
160 packets received by filter
0 packets dropped by kernel

# file test.gz 
test.gz: gzip compressed data, last modified: Wed May 23 12:56:16 2018, from Unix
# 

在 Debian 9（Stretch）上工作正常，但在 Debian 8（Jessie）上却没问题：

This works fine on Debian 9 (Stretch) but not on Debian 8 (Jessie):

# tcpdump -l -i eth0 | ( gzip -c > test.gz & )
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
tcpdump: Unable to write output: Broken pipe
# 

两个问题：

1. 显而易见的解决方案出了什么问题？


2. 如何捕获并压缩Debian Jessie中的tcpdump输出？ （明显的解决方案也不起作用）

谢谢！

推荐答案

发生了什么

解释在这里发生的情况：

What Was Happening

To explain what happens here:

• Ctrl + C 发送SIGINT到整个过程组。这意味着它不仅终止 tcpdump ，而且终止 gzip 。 （您尝试通过将内容移入后台进程，从而移出同一进程组来避免这种情况的解决方法。）


• stdout默认仅在输出为TTY；当输出到FIFO时，将对其进行块缓冲，只有在有足够大的块可用时，才通过写入左侧过程中的数据来提高效率。因此，在许多情况下，您可以使用 stdbuf -oL 或类似的方法禁用此功能。但是...


• gzip 就其性质而言无法完全无缓冲地运行。这是因为基于块的压缩算法需要将数据收集到 block 中；批量分析该内容； & c。

• Ctrl+C sends a SIGINT to the entire process group. That means it doesn't just terminate tcpdump, but also terminates gzip. (The workarounds you were attempting try to avoid this by moving content into background processes, and thus out of the same process group).
• stdout is line-buffered by default only when output is to a TTY; when output is to a FIFO, it's block-buffered, allowing greater efficiency by writing data from the left-hand process only when a sufficiently larger chunk is available. In many situations, you could thus just use stdbuf -oL or similar to disable this. However...
• gzip by its nature cannot operate completely unbuffered. This is because block-based compression algorithms need to collect data into, well, blocks; analyze that content in bulk; &c.

因此，如果 gzip 和 tcpdump 同时终止，这意味着不能保证 tcpdump 实际上将能够刷新其输出缓冲区和然后让 gzip 读取，压缩和写入该刷新数据，然后在 gzip 自身退出信号之前

So, if gzip and tcpdump are terminated at the same time, that means there's no assurance that tcpdump will actually be able to flush its output buffer, and then have gzip read, compress and write that flushed data, before gzip itself exits from the signal it received at the same time.

请注意，包含单词 Interactive的标题下的代码段旨在用于 interactive use 。

Note that the code snippets under headers containing the word "Interactive" are intended for interactive use.

作为surefire解决方案，将 gzip 完全移出带，因此当您在 tcpdump 命令上按ctrl + c时，它不容易发送SIGINT：

As a surefire solution, move the gzip completely out-of-band, so it isn't prone to being sent a SIGINT when you press ctrl+c on the tcpdump command:

exec 3> >(gzip -c >test.gz)  # Make FD 3 point to gzip
tcpdump -l -i eth0 >&3       # run tcpdump **AS A SEPARATE COMMAND** writing to that fd
exec 3>&-                    # later, after you cancelled tcpdump, close the FD.

---

可靠的交互式解决方法（适用于任何POSIX Shell）

同样的东西，但是稍长一些并且不依赖进程替换：

---

A Reliable Interactive Workaround (For Any POSIX Shell)

Same thing, but slightly longer and not relying on process substitution:

mkfifo test.fifo                            # create a named FIFO
gzip -c <test.fifo >test.gz & gzip_pid="$!" # start gzip, reading from that named FIFO
tcpdump -l -i eth0 >test.fifo               # start tcpdump, writing to that named FIFO
rm test.fifo                                # delete the FIFO when done
wait "$gzip_pid"                            # ...and wait for gzip to exit

请注意 wait 将具有gzip进程的退出状态，因此您可以确定它是否遇到错误。

Note that the wait will have the exit status of the gzip process, so you can determine whether it encountered an error.

如果我们正在运行脚本，则设置信号处理程序是适当的因此我们可以显式处理SIGINT（通过杀死 tcpdump）：

If we're running a script, then it's appropriate to set up a signal handler so we can handle SIGINT (by killing only tcpdump) explicitly:

#!/bin/sh
[ "$#" -gt 0 ] || {
  echo "Usage: ${0##*/} file.tcpdump.gz [tcpdump-args]" >&2
  echo "  Example: ${0##*/} foo.tcpdump.gz -l -i eth0" >&2
  exit 1
}
outfile=$1; shift
fifo=test-$$.fifo # for real code, put this in a unique temporary directory

trap '[ -n "$tcpdump_pid" ] && kill "$tcpdump_pid"' INT
trap 'rm -f -- "$fifo"' EXIT

rm -f -- "$fifo"; mkfifo "$fifo" || exit
gzip -c >"$outfile" <"$fifo" & gzip_pid=$!

# avoid trying to run tcpdump if gzip obviously failed to start
{ [ -n "$gzip_pid" ] && [ "$gzip_pid" -gt 0 ] && kill -0 "$gzip_pid"; } || exit 1

tcpdump "$@" >"$fifo" & tcpdump_pid=$!

# return exit status of tcpdump if it fails, or gzip if tcpdump succeeds
wait "$tcpdump_pid" || wait "$gzip_pid"

这篇关于将tcpdump输出写入压缩/ gzip压缩文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！