按照模式将大文件分成小文件 [英] Divide very large file into small ones following pattern

问题描述

我一直在解决这个问题上，但是收效甚微，所以我来这里是为了获得一些新的建议.

I have been working on this problem with only little success so I am coming here to get some fresh advices.

我正在尝试将每次扫描的数据提取到单独的文件中.

I am trying to extract the data of every scan into separate files.

问题在于，在创建3196个文件之后，我收到错误消息:awk制作了太多打开的文件".

The problem is that after 3196 files created I receive the error message : awk "makes too many open files".

我知道我需要关闭awk创建的文件，但是我不知道该怎么做.

I understand that I need to close the files created by awk but I don't know how to do that.

文本输入文件看起来像这样(最多80 000个扫描):

Text inputfile is looking like this (up to 80 000 Scan):

Scan    1
11111    111
22222    221
...
Scan    2
11122    111
11122    111
...
Scan    3
11522    141
19922    141
...

现在我一直在做:

awk '/.*Scan.*/{n++}{print >"filescan" n }' inputfile

在创建3196个文件后，每次扫描和崩溃都会给我增加的输出文件.

Which gives me an incremented output file for every Scan and crash after 3196 files created..

cat filescan1
Scan    1
11111    111
22222    221
...

有什么主意吗?

推荐答案

您需要关闭输出文件，因为awk使文件句柄保持打开状态.

You need to close the output file as awk is keeping the file handle open.

awk '/.*Scan.*/{ 
  close(file);
  n++;
}
{ 
  file="filescan"n; 
  print >> file;
}' inputfile

这篇关于按照模式将大文件分成小文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！