如何在文件中每一行的开头均匀添加一组值? [英] How to evenly add a set of values at the beginning of each line in a file?

问题描述

我想使用bash或awk或其他任何方法来执行以下操作.我有一个看起来像这样的文件:

I'd like to use bash or awk or anything to do the following. I have a file that looks like:

/exec/proc1 ...
/exec/proc1 ...
/exec/proc1 ...
/exec/proc1 ...
/exec/proc1 ...
/exec/proc1 ...

如果我说3，即(0，1，2)，我想生成一个文件，例如:

and if I say 3 i.e. (0, 1, 2) I'd like to produce a file like:

export DEVICE_ID=0; /exec/proc1 ...
export DEVICE_ID=0; /exec/proc1 ...
export DEVICE_ID=1; /exec/proc1 ...
export DEVICE_ID=1; /exec/proc1 ...
export DEVICE_ID=2; /exec/proc1 ...
export DEVICE_ID=2; /exec/proc1 ...

然后将其通过管道传送到shuf中，这将使行随机化.这是在多个CUDA GPU设备之间实现负载平衡的穷人解决方案.

I then pipe this into shuf that will randomize the lines. This is a poor's man solution to load balance across multiple CUDA GPU devices.

推荐答案

使用awk，您可以执行以下操作:

Using awk you can do:

awk -v n=3 '{print "export DEVICE_ID=" (NR-1)%n "; " $0}' file
export DEVICE_ID=0; /exec/proc1 ...
export DEVICE_ID=1; /exec/proc1 ...
export DEVICE_ID=2; /exec/proc1 ...
export DEVICE_ID=0; /exec/proc1 ...
export DEVICE_ID=1; /exec/proc1 ...
export DEVICE_ID=2; /exec/proc1 ...
export DEVICE_ID=0; /exec/proc1 ...
export DEVICE_ID=1; /exec/proc1 ...
export DEVICE_ID=2; /exec/proc1 ...
export DEVICE_ID=0; /exec/proc1 ...

这篇关于如何在文件中每一行的开头均匀添加一组值?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！