删除重复的行仅包含特定的字符串 [英] Remove duplicate line only contain specific string
本文介绍了删除重复的行仅包含特定的字符串的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
仅在包含特定字符串的情况下,我才尝试删除重复的行.删除重复的行很容易,但是一些有用的行会用:
I try to remove duplicates lines only if contain a specific string. It's easy to remove only duplicates lines, but some useful lines is deleted with :
awk '!seen[$0]++'
或
perl -ne 'print unless $seen{$_}++'
示例:
保留包含"host_name ="
的行的第一次出现保留所有包含"plugin output ="
上面的 awk
或 Perl
命令也删除了客户端编号.
keep first occurence of lines containing "host_name="
keep all occurrences of lines containing "plugin output="
with above awk
or Perl
commands that delete the client number too.
我的输出命令:
host_name=Client1
plugin_output=Name : Client1 Marseille
host_name=Client1
plugin_output=Client : 168131
host_name=Client1
host_name=Client1
host_name=Client1
host_name=Client1
host_name=Client1
host_name=Client1
host_name=Client1
host_name=Client2
plugin_output=Name : Client2 Besançon
host_name=Client2
plugin_output=Client : 168131
host_name=Client2
host_name=Client2
host_name=Client2
host_name=Client2
host_name=Client2
host_name=Client2
host_name=Client2
host_name=Client2
host_name=Client3
plugin_output=Name : Client3 BRETAGNE
host_name=Client3
plugin_output=Client : 168131
host_name=Client3
host_name=Client3
host_name=Client3
host_name=Client3
host_name=Client3
host_name=Client3
host_name=Client3
host_name=Client4
plugin_output=Name : Client4
host_name=Client4
plugin_output=Client : 168131
host_name=Client4
host_name=Client4
host_name=Client4
host_name=Client4
host_name=Client4
host_name=Client4
host_name=Client4
host_name=Client5
plugin_output=Name : Client5
host_name=Client5
plugin_output=Client : 168131
host_name=Client5
host_name=Client5
host_name=Client5
host_name=Client5
host_name=Client5
host_name=Client5
host_name=Client5
host_name=Client6
plugin_output=Name : Client6
host_name=Client6
plugin_output=Client : 168131
host_name=Client6
host_name=Client6
host_name=Client6
host_name=Client6
host_name=Client6
host_name=Client6
host_name=Client6
host_name=Client6
host_name=Client7
plugin_output=Name : Client7
host_name=Client7
plugin_output=Client : 168131
host_name=Client7
host_name=Client7
host_name=Client7
host_name=Client7
host_name=Client7
host_name=Client7
host_name=Client7
host_name=Client7
host_name=Client8
plugin_output=Name : Client8
host_name=Client8
plugin_output=Client : 168131
host_name=Client8
host_name=Client8
host_name=Client8
host_name=Client8
host_name=Client8
host_name=Client8
host_name=Client8
awk/sed/perl/...之后的期望值:
Desired value after awk/sed/perl/...:
host_name=Client1
plugin_output=Name : Client1 Marseille
plugin_output=Client : 168131
host_name=Client2
plugin_output=Name : Client2 Besançon
plugin_output=Client : 168131
host_name=Client3
plugin_output=Name : Client3 BRETAGNE
plugin_output=Client : 168131
host_name=Client4
plugin_output=Name : Client4
plugin_output=Client : 168131
host_name=Client5
plugin_output=Name : Client5
plugin_output=Client : 168131
host_name=Client6
plugin_output=Name : Client6
plugin_output=Client : 168131
host_name=Client7
plugin_output=Name : Client7
plugin_output=Client : 168131
host_name=Client8
plugin_output=Name : Client8
plugin_output=Client : 168131
推荐答案
您可以使用以下awk:
You can use this awk:
awk '/^plugin_output=/ || !seen[$0]++' file
host_name=Client1
plugin_output=Name : Client1 Marseille
plugin_output=Client : 168131
host_name=Client2
plugin_output=Name : Client2 Besançon
plugin_output=Client : 168131
host_name=Client3
plugin_output=Name : Client3 BRETAGNE
plugin_output=Client : 168131
host_name=Client4
plugin_output=Name : Client4
plugin_output=Client : 168131
host_name=Client5
plugin_output=Name : Client5
plugin_output=Client : 168131
host_name=Client6
plugin_output=Name : Client6
plugin_output=Client : 168131
host_name=Client7
plugin_output=Name : Client7
plugin_output=Client : 168131
host_name=Client8
plugin_output=Name : Client8
plugin_output=Client : 168131
如果记录以 plugin_output =
开头或唯一,则打印该记录.
Which prints a record if it starts with plugin_output=
or if it is unique.
这篇关于删除重复的行仅包含特定的字符串的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文