使用 unix 实用程序删除连续的重复行 [英] Delete consecutive duplicate lines using unix utilities

问题描述

这听起来很简单，但实际上要复杂一些.我想使用一个unix实用程序来删除连续的重复项，留下原来的.但是，我还想保留在原始文件之后不会立即出现的其他副本.例如，如果我们有以下行:

This sounds simple on its face but is actually somewhat more complex. I would like to use a unix utility to delete consecutive duplicates, leaving the original. But, I would also like to preserve other duplicates that do not occur immediately after the original. For example, if we have the lines:

O B 
O B 
C D 
T V
O B

我希望输出是:

O B 
C D
T V
O B 

虽然第一行和最后一行相同，但它们不是连续的，因此我想将它们保留为唯一条目.

Although the first and last lines are the same, they are not consecutive and therefore I want to keep them as unique entries.

推荐答案

你可以这样做:

cat file1 | uniq > file2

或更简洁地说:

uniq file1 file2

假设 file1 包含

O B
O B
C D
T V
O B

有关详细信息，请参阅 man uniq.特别要注意，uniq 命令接受两个参数，语法如下:uniq [OPTION]... [INPUT [OUTPUT]].

For more details, see man uniq. In particular, note that the uniq command accepts two arguments with the following syntax: uniq [OPTION]... [INPUT [OUTPUT]].

最后，如果您想删除所有重复项(并对文件进行排序)，您可以这样做:

Finally if you'd want to remove all duplicates (and sort the file along the way), you could do:

sort -u file1 > file2

这篇关于使用 unix 实用程序删除连续的重复行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！