将 tcpdump 输出写入压缩/gzip 文件 [英] Write tcpdump output to compressed / gziped file

问题描述

我想将 tcpdump 的文本输出写入压缩文件.

I want to write the textual output of tcpdump to a compressed file.

首先我尝试了最明显的:

First I tried the most obvious:

# tcpdump -l -i eth0 | gzip -c > test.gz
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C63 packets captured
244 packets received by filter
0 packets dropped by kernel
4 packets dropped by interface

# file test.gz
test.gz: empty
# 

然后我找到了 Debian 9 (Stretch) 的以下解决方案:

Then I found the following solution for Debian 9 (Stretch):

# tcpdump -l -i eth0 | ( gzip -c > test.gz & )
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C150 packets captured
160 packets received by filter
0 packets dropped by kernel

# file test.gz 
test.gz: gzip compressed data, last modified: Wed May 23 12:56:16 2018, from Unix
# 

这适用于 Debian 9 (Stretch) 但不适用于 Debian 8 (Jessie):

This works fine on Debian 9 (Stretch) but not on Debian 8 (Jessie):

# tcpdump -l -i eth0 | ( gzip -c > test.gz & )
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
tcpdump: Unable to write output: Broken pipe
# 

两个问题:

1. 显而易见的解决方案"有什么问题?
2. 如何在 Debian Jessie 中捕获和压缩 tcpdump 输出?(显而易见的解决方案也不起作用)

谢谢！

推荐答案

发生了什么

解释这里发生了什么:

What Was Happening

To explain what happens here:

• Ctrl+C 向整个进程组发送一个 SIGINT.这意味着它不仅会终止 tcpdump，还会终止 gzip.(您尝试的解决方法是通过将内容移到后台进程中来避免这种情况，从而移出同一个进程组).
• 默认情况下，stdout 仅在输出到 TTY 时才进行行缓冲；当输出到 FIFO 时，它是块缓冲的，只有在足够大的块可用时才通过从左侧进程写入数据来提高效率.在许多 的情况下，您可以因此只使用 stdbuf -oL 或类似的来禁用它.不过……
• gzip 本质上不能完全无缓冲地运行.这是因为基于块的压缩算法需要将数据收集到块中；批量分析该内容；&c.

• Ctrl+C sends a SIGINT to the entire process group. That means it doesn't just terminate tcpdump, but also terminates gzip. (The workarounds you were attempting try to avoid this by moving content into background processes, and thus out of the same process group).
• stdout is line-buffered by default only when output is to a TTY; when output is to a FIFO, it's block-buffered, allowing greater efficiency by writing data from the left-hand process only when a sufficiently larger chunk is available. In many situations, you could thus just use stdbuf -oL or similar to disable this. However...
• gzip by its nature cannot operate completely unbuffered. This is because block-based compression algorithms need to collect data into, well, blocks; analyze that content in bulk; &c.

所以，如果 gzip 和 tcpdump 同时终止，这意味着无法保证 tcpdump 实际上能够刷新它的输出缓冲区，然后让 gzip 读取、压缩和写入刷新的数据，然后 gzip 本身退出它同时接收到的信号时间.

So, if gzip and tcpdump are terminated at the same time, that means there's no assurance that tcpdump will actually be able to flush its output buffer, and then have gzip read, compress and write that flushed data, before gzip itself exits from the signal it received at the same time.

请注意，包含交互式"一词的标题下的代码片段旨在交互式使用.

Note that the code snippets under headers containing the word "Interactive" are intended for interactive use.

作为一个万无一失的解决方案，将 gzip 完全移出带外，这样当您在 tcpdump 上按 ctrl+c 时它不容易被发送 SIGINT 代码>命令:

As a surefire solution, move the gzip completely out-of-band, so it isn't prone to being sent a SIGINT when you press ctrl+c on the tcpdump command:

exec 3> >(gzip -c >test.gz)  # Make FD 3 point to gzip
tcpdump -l -i eth0 >&3       # run tcpdump **AS A SEPARATE COMMAND** writing to that fd
exec 3>&-                    # later, after you cancelled tcpdump, close the FD.

<小时>

可靠的交互式解决方法(适用于任何 POSIX Shell)

同样的事情，但稍长一些，并且不依赖于进程替换:

---

A Reliable Interactive Workaround (For Any POSIX Shell)

Same thing, but slightly longer and not relying on process substitution:

mkfifo test.fifo                            # create a named FIFO
gzip -c <test.fifo >test.gz & gzip_pid="$!" # start gzip, reading from that named FIFO
tcpdump -l -i eth0 >test.fifo               # start tcpdump, writing to that named FIFO
rm test.fifo                                # delete the FIFO when done
wait "$gzip_pid"                            # ...and wait for gzip to exit

注意wait会有gzip进程的退出状态，可以判断是否遇到错误.

Note that the wait will have the exit status of the gzip process, so you can determine whether it encountered an error.

如果我们正在运行一个脚本，那么设置一个信号处理程序是合适的，这样我们就可以显式地处理 SIGINT(通过杀死 only tcpdump):

If we're running a script, then it's appropriate to set up a signal handler so we can handle SIGINT (by killing only tcpdump) explicitly:

#!/bin/sh
[ "$#" -gt 0 ] || {
  echo "Usage: ${0##*/} file.tcpdump.gz [tcpdump-args]" >&2
  echo "  Example: ${0##*/} foo.tcpdump.gz -l -i eth0" >&2
  exit 1
}
outfile=$1; shift
fifo=test-$$.fifo # for real code, put this in a unique temporary directory

trap '[ -n "$tcpdump_pid" ] && kill "$tcpdump_pid"' INT
trap 'rm -f -- "$fifo"' EXIT

rm -f -- "$fifo"; mkfifo "$fifo" || exit
gzip -c >"$outfile" <"$fifo" & gzip_pid=$!

# avoid trying to run tcpdump if gzip obviously failed to start
{ [ -n "$gzip_pid" ] && [ "$gzip_pid" -gt 0 ] && kill -0 "$gzip_pid"; } || exit 1

tcpdump "$@" >"$fifo" & tcpdump_pid=$!

# return exit status of tcpdump if it fails, or gzip if tcpdump succeeds
wait "$tcpdump_pid" || wait "$gzip_pid"

这篇关于将 tcpdump 输出写入压缩/gzip 文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！