从无休止的管庆典阅读 [英] Read from a endless pipe bash

问题描述

我想创建一个运行的每个第一个接收管道线路的其它脚本的脚本。
像这样的：

I want to create a script that runs another script for each line that the first one receives piping. Like this:

journalctl -f | myScript1.sh

本 myScript1.sh 将运行另外一个是这样的：

this myScript1.sh will run another one like this:

./myScript2.sh $line_in_pipe

问题，我发现我是每一个测试code只是在一个有限的管道运行良好（直到 EOF ）。
但是，当像尾-f 或其它管我的方案，它只是将不会执行。我认为它只是等待 EOF 来执行循环。

Problem I found is every code I tested just runs well in a finite pipe (till EOF). But when I pipe programs like tail -f or others it just won't execute. I think it just waits for EOF to execute the loop.

编辑：
无尽的管道是这样的：

the endless pipe is like this:

tail -f /var/log/apache2/access.log | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | script_ip_check.sh

等script_ip_check.sh的想法是做这样的事情：

so the idea on script_ip_check.sh is doing something like this:

#!/bin/bash

for line in $(cat); do
        echo "process:$line"
        nmap -sV -p1234 --open -T4 $line | grep 'open' -B3 | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' >> list_of_ip_mapped &
done

每一行，这种情况下IP，我将产生NMAP的线程来扫描一些主机上特别。
我将用它来扫描试图连接我的服务器上的一些隐藏的端口IP地址。
所以，我一定要运行脚本，直到我取消它，或者它接收到EOF的所有时光。

for each line, this case IP, I will spawn a thread of nmap to scan something special on that host. I will use it to scan IPs that tries to connect some "hidden" port on my server. So my script must runs all the time till I cancel it or it receives an EOF.

EDIT2：
我刚刚发现的grep 刷新其缓冲区所以这就是为什么它不工作。
我我使用 - 行缓冲强制的grep 来输出，因为它是被处理每行

I just found out that grep flushes its buffer so that's why it's not working. I I use --line-buffered to force grep to output each line as it's being processed.

推荐答案

我们还不能确切地说不知道什么是在你的脚本。

We can't say definitively without knowing what's in your script.

举例来说，如果你的目的是：

For instance, if you're doing this:

# DON'T DO THIS: Violates http://mywiki.wooledge.org/DontReadLinesWithFor
for line in $(cat); do
  : ...do something with "$line"...
done

...这要等到所有的标准输入可用，导致挂你描述。

...that'll wait until all stdin is available, resulting in the hang you describe.

不过，如果你遵循最佳实践（每 BashFAQ＃1 ），你的$ C $按c更多类似这样的操作：

However, if you're following best practices (per BashFAQ #1), your code will operate more like this:

while IFS= read -r line; do
  : ...do something with "$line"
done

...这实际上会循规蹈矩，的受作家进行任何缓冲的。有关控制缓冲的提示，请参阅 BashFAQ＃9 。

...and that'll actually behave properly, subject to any buffering performed by the writer. For hints on controlling buffering, see BashFAQ #9.

最后，从 DontReadLinesWithFor 报价：

伴读线，为的最后一个问题是效率低下。 A 而读循环读取从输入流一次一行; $（小于å文件）吸食整个文件到内存中的一次。对于小文件，这不是一个问题，但如果你正在阅读大量文件，对内存的需求将是巨大的。 （击将分配一个字符串来保存文件，而另一组字符串来保存字拆分结果...基本上，分配的内存将被输入文件大小的两倍）。

The final issue with reading lines with for is inefficiency. A while read loop reads one line at a time from an input stream; $(<afile) slurps the entire file into memory all at once. For small files, this is not a problem, but if you're reading large files, the memory requirement will be enormous. (Bash will have to allocate one string to hold the file, and another set of strings to hold the word-split results... essentially, the memory allocated will be twice the size of the input file.)

显然，如果内容是不定，存储器要求和完成时间同样是

Obviously, if the content is indefinite, the memory requirements and completion time are likewise.

这篇关于从无休止的管庆典阅读的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！