在for循环中执行awk命令以读取和写入多个文件后无法关闭 [英] Can't close after awk command inside for loop to read and write multiple files

问题描述

我有一组csv文件，其中一个具有时间戳，所有其他仅具有数据，但是它们都具有相同的行数.时间戳记在csv的第二列中.

I have a set of csv files, one of them has a timestamp, all the other ones just have data, but they all have the same number of rows. The timestamp is in the 2nd column of the csv.

我想将时间戳添加到所有csv文件的第一列.这目前正在工作，但是当我尝试关闭文件时，出现错误.可以有50-500个csv文件，每个文件可以有数千行，所以这就是为什么我想知道是否需要close()的原因.

I would like to append the timestamp to the first column of all csv files. This is currently working but when I try to close the files, I get an error. There can be 50-500 csv files, and each can have thousands of rows, so that is why I wonder if the close() is required.

还有，有人可以提出任何方法来改善此脚本的性能或可靠性，并检查是否存在任何错误吗?

Also, can anyone suggest any ways of improving this script either for performance or for reliability and check for any errors?

样本输入blah_blah_blah_timestamp.csv

Sample input blah_blah_blah_timestamp.csv

name, time
1,121
2,122
3,123

data1.csv

data1.csv

name,X1
A1,11
A2,12
A3,15

data2.csv

data2.csv

name,Y1,Y2,Y3
B1,1,1,2
B2,2,1,3
B3,3,2,4

data3.csv

data3.csv

name,Z1,Z2,Z3
C1,1,9,5
C2,2,8,4
C3,3,7,3

out/data1.csv

out/data1.csv

time,name,X1
121,A1,11
122,A2,12
123,A3,15

out/data2.csv

out/data2.csv

time,name,Y1,Y2,Y3
121,B1,1,1,2
122,B2,2,1,3
123,B3,3,2,4

out/data3.csv

out/data3.csv

time,name,Z1,Z2,Z3
121,C1,1,9,5
122,C2,2,8,4
123,C3,3,7,3

当前脚本

#!/bin/bash
mkdir -p out
ts='blah_blah_blah_timestamp.csv'
for sfile in *.csv;
do
   awk -F, -v afile="$sfile" '{getline f1 < afile ;print $2, f1}' OFS=, $ts >  out/"$sfile"
   close('sfile')
done

推荐答案

这可能是您想要的:

awk '
BEGIN { FS=OFS="," }
NR==FNR {
    map[FNR] = $2
    next
}
FNR==1 {
    close(out)
    out = "out/" FILENAME
}
{ print map[FNR], $0 > out }
' blah_blah_blah_timestamp.csv data*.csv

请参阅 https://stackoverflow.com/a/65814521/1745001 ，了解有关我为什么/如何做的说明m使用 close().

See https://stackoverflow.com/a/65814521/1745001 for an explanation for why/how I'm using close().

顺便说一句，如果您将来考虑使用 getline -这不是正确的方法，而且很难正确使用，请参见

By the way, if you're ever considering using getline in future - it's rarely the right approach and it's hard to use correctly, see http://awk.freeshell.org/AllAboutGetline.

这篇关于在for循环中执行awk命令以读取和写入多个文件后无法关闭的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！