如何用grep code的输出唯一的行从一个文件？ [英] How to use grep to output unique lines of code from a file?

问题描述

我有一个包含线，如一个大的日志文件：

I have a large log file that contains lines such as:

82.117.22.206 - - [08/Mar/2013:20:36:42 +0000] "GET /key/0/www.mysite.org.uk/ HTTP/1.0" 200 0 "-" "-"

和我想从每一个上面的图案匹配的行只提取IP 82.117.22.206 后跟一个空格和文本 WWW。 mysite.org.uk 从它。的IP地址和文本可以不同。因此，鉴于上述线路输出文件的行会是：

And i want to extract from each line that matches the above pattern only the ip 82.117.22.206 followed by a space and the text www.mysite.org.uk from it. The ip and text can differ. So given the above line the line in the output file would be:

82.117.22.206 www.mysite.org.uk

我如何使用grep或其他命令在bash使输出唯一的，这样的输出文件将不包含两个相同的行？有人可以参考我一个良好的开端之学更多关于这种shell脚本的？

How can I use grep or other commands in bash to make the output unique so that the output file won't contain two identical lines? Can someone refer me to a good place to start learnning more about this kind of shell scripting?

推荐答案

用Perl可以捕捉到部分

With perl you can capture the parts

use strict;
use warnings;

if (m/^(\d+\.\d+\.\d+\.\d+)\s+-\s+-\s+\[.+?\]\s+\"GET\s+\/key\/0\/(.+?)\//) {
    print "$1 $2\n";
}

和调用此为

perl -n script.pl logfile.txt | sort -u

这提取所需的字段，排序和消除了重复行。

This extracts the needed fields, sorts and eliminates duplicate lines.

这篇关于如何用grep code的输出唯一的行从一个文件？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！