在 unix 命令行中删除文件的前 N ​​行 [英] Remove first N lines of a file in place in unix command line

问题描述

我正在尝试从一个非常非常大的文件中删除前 37 行.我开始尝试 sed 和 awk，但它们似乎需要将数据复制到新文件中.我正在寻找一种就地删除行"的方法，与 sed -i 不同的是，它不会制作任何类型的副本，而只是从现有文件中删除行.

I'm trying to remove the first 37 lines from a very, very large file. I started trying sed and awk, but they seem to require copying the data to a new file. I'm looking for a "remove lines in place" method, that unlike sed -i is not making copies of any kind, but rather is just removing lines from the existing file.

这就是我所做的......

Here's what I've done...

awk 'NR > 37' file.xml > 'f2.xml'
sed -i '1,37d' file.xml

这两个似乎都做了一个完整的副本.有没有其他简单的 CLI 可以在不遍历完整文档的情况下快速完成此操作?

Both of these seem to do a full copy. Is there any other simple CLI that can do this quickly without a full document traversal?

推荐答案

没有使用 UNIX 实用程序进行就地编辑的简单方法，但这里有一个就地文件修改解决方案，您可以对其进行修改以适合您(由罗伯特博诺米在 https://groups.google.com/forum/#!topic/comp.unix.shell/5PRRZIP0v64):

There's no simple way to do inplace editing using UNIX utilities, but here's one inplace file modification solution that you might be able to modify to work for you (courtesy of Robert Bonomi at https://groups.google.com/forum/#!topic/comp.unix.shell/5PRRZIP0v64):

bytes=$(head -37 "$file" |wc -c)
dd if="$file" bs="$bytes" skip=1 conv=notrunc of="$file"

最终文件应该比原始文件小 $bytes 个字节(因为目标是从开头删除 $bytes 个字节)，所以要完成我们必须删除最后的 $bytes 字节.我们在上面使用 conv=notrunc 来确保文件不会被完全清空，而不仅仅是被截断(例如，请参见下文).在 GNU 系统(例如 Linux)上，可以通过以下方式完成截断:

The final file should be $bytes bytes smaller than the original (since the goal was to remove $bytes bytes from the beginning), so to finish we must remove the final $bytes bytes. We're using conv=notrunc above to make sure that the file doesn't get completely emptied rather than just truncated (see below for example). On a GNU system such as Linux doing the truncation afterwards can be accomplished by:

truncate -s "-$bytes" "$file"

例如从这个 12 行文件中删除前 5 行

For example to delete the first 5 lines from this 12-line file

$ wc -l file
12 file

$ cat file
When chapman billies leave the street,
And drouthy neibors, neibors, meet;
As market days are wearing late,
And folk begin to tak the gate,
While we sit bousing at the nappy,
An' getting fou and unco happy,
We think na on the lang Scots miles,
The mosses, waters, slaps and stiles,
That lie between us and our hame,
Where sits our sulky, sullen dame,
Gathering her brows like gathering storm,
Nursing her wrath to keep it warm.

首先使用 dd 从文件的开头删除目标 5 行(实际上是$bytes"字节)，然后将其余部分从文件末尾复制到前面，但保留尾随的$bytes"字节.$bytes"字节原样:

First use dd to remove the target 5 lines (really "$bytes" bytes) from the start of the file and copy the rest from the end to the front but leave the trailing "$bytes" bytes as-is:

$ bytes=$(head -5 file |wc -c)

$ dd if=file bs="$bytes" skip=1 conv=notrunc of=file
1+1 records in
1+1 records out
253 bytes copied, 0.0038458 s, 65.8 kB/s

$ wc -l file
12 file

$ cat file
An' getting fou and unco happy,
We think na on the lang Scots miles,
The mosses, waters, slaps and stiles,
That lie between us and our hame,
Where sits our sulky, sullen dame,
Gathering her brows like gathering storm,
Nursing her wrath to keep it warm.
s, waters, slaps and stiles,
That lie between us and our hame,
Where sits our sulky, sullen dame,
Gathering her brows like gathering storm,
Nursing her wrath to keep it warm.

然后使用 truncate 从末尾删除那些剩余的字节:

and then use truncate to remove those leftover bytes from the end:

$ truncate -s "-$bytes" "file"

$ wc -l file
7 file

$ cat file
An' getting fou and unco happy,
We think na on the lang Scots miles,
The mosses, waters, slaps and stiles,
That lie between us and our hame,
Where sits our sulky, sullen dame,
Gathering her brows like gathering storm,
Nursing her wrath to keep it warm.

如果我们在没有 dd ... conv=notrunc 的情况下尝试了上述:

If we had tried the above without dd ... conv=notrunc:

$ wc -l file
12 file
$ bytes=$(head -5 file |wc -c)
$ dd if=file bs="$bytes" skip=1 of=file
dd: file: cannot skip to specified offset
0+0 records in
0+0 records out
0 bytes copied, 0.0042254 s, 0.0 kB/s
$ wc -l file
0 file

有关其他建议和信息，请参阅我引用的 google 群组主题.

See the google groups thread I referenced for other suggestions and info.

这篇关于在 unix 命令行中删除文件的前 N ​​行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！