在第N个分隔符出现时分割文件 [英] split file on Nth occurrence of delimiter

问题描述

在每次 第N次出现 定界符之后，是否有一个直线将文本文件拆分为小块/大块？

Is there a one-liner to split a text file into pieces / chunks after every Nth occurrence of a delimiter?

示例：下面的定界符为 +

example: the delimiter below is "+"

entry 1
some more
+
entry 2
some more
even more
+
entry 3
some more
+
entry 4
some more
+
...

有几百万个条目，因此，对每次出现的定界符 +进行拆分都是一个坏主意。我想分割每个定界符 +的第50,000个实例。

There are several million entries, so splitting on every occurrence of delimiter "+" is a bad idea. I want to split on, say, every 50,000th instance of delimiter "+".

Unix命令 split和 csplit似乎并不执行此操作...

Unix commands "split" and "csplit" just don't seem to do this...

推荐答案

使用 awk 您可以：

awk '/^\+$/ { delim++ } { file = sprintf("chunk%s.txt", int(delim / 50000)); print >> file; }' < input.txt 

更新：

到不包括定界符，请尝试以下操作：

To not include the delimiter, try this:

awk '/^\+$/ { if(++delim % 50000 == 0) { next } } { file = sprintf("chunk%s.txt", int(delim / 50000)); print > file; }' < input.txt 

next 关键字会导致awk暂停此记录的处理规则，并前进到下一个（行）。我也将>> 更改为> ，因为如果多次运行它，可能就不会想要附加旧的块文件。

The next keyword causes awk to halt processing rules for this record and and advance to the next (line). I also changed the >> to > since if you run it more than once you probably don't want to append the old chunk files.

这篇关于在第N个分隔符出现时分割文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！