在第 N 次出现分隔符时拆分文件 [英] split file on Nth occurrence of delimiter

问题描述

在每 N 次出现 分隔符之后，是否有一个单行将文本文件拆分为多个片段/块?

Is there a one-liner to split a text file into pieces / chunks after every Nth occurrence of a delimiter?

示例:下面的分隔符是+"

example: the delimiter below is "+"

entry 1
some more
+
entry 2
some more
even more
+
entry 3
some more
+
entry 4
some more
+
...

有几百万个条目，因此在每次出现分隔符+"时进行拆分是一个坏主意.例如，我想拆分分隔符+"的第 50,000 个实例.

There are several million entries, so splitting on every occurrence of delimiter "+" is a bad idea. I want to split on, say, every 50,000th instance of delimiter "+".

Unix 命令 "split" 和 "csplit" 似乎没有这样做......

Unix commands "split" and "csplit" just don't seem to do this...

推荐答案

使用 awk 你可以:

awk '/^+$/ { delim++ } { file = sprintf("chunk%s.txt", int(delim / 50000)); print >> file; }' < input.txt 

更新:

要不包含分隔符，请尝试以下操作:

To not include the delimiter, try this:

awk '/^+$/ { if(++delim % 50000 == 0) { next } } { file = sprintf("chunk%s.txt", int(delim / 50000)); print > file; }' < input.txt 

next 关键字使 awk 停止处理此记录的规则并前进到下一行(行).我还将 >> 更改为 > 因为如果您多次运行它，您可能不想附加旧的块文件.

The next keyword causes awk to halt processing rules for this record and and advance to the next (line). I also changed the >> to > since if you run it more than once you probably don't want to append the old chunk files.

这篇关于在第 N 次出现分隔符时拆分文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！