带尾-n读行时如何使用 [英] How to use while read line with tail -n

问题描述

问题:我有一个CSV转储文件-超过250,000行.当我使用while read时-需要一段时间(没有双关语).我想回到最后的10,000行来完成我需要做的事情，而不是250,000行.

Problem: I have a CSV dump file - with excess of 250,000 lines. When I use while read - it takes a while (no pun intended). I would like to go back to the last 10,000 lines to do what I need to do instead of the 250,000 lines.

代码段:我当前的代码是:

IFS=","
while read line
do

    awk_var=`echo "$line" | awk -F" " '{print $0}'`

    var_array=($awk_var)

    read -a var_array <<< "${awk_var}"

    echo "${var_array[1]}"


done </some_directory/directory/file_in_question.csv

问题:当使用bash脚本读取file_in_question.csv时，如何在while read line中使用tail -n10000?

Question: How can I use tail -n10000 with while read line when reading the file_in_question.csv with a bash script?

推荐答案

替换:

done </some_directory/directory/file_in_question.csv

具有:

done < <(tail -n10000 /some_directory/directory/file_in_question.csv)

<(...)构造称为进程替换.它创建了bash可以读取的类似文件的对象.因此，这将直接从tail -n10000 /some_directory/directory/file_in_question.csv读取替换为从some_directory/directory/file_in_question.csv读取.

The <(...) construct is called process substitution. It creates a file-like object that bash can read from. Thus, this replaces reading from some_directory/directory/file_in_question.csv directly with reading from tail -n10000 /some_directory/directory/file_in_question.csv.

使用这样的进程替换，可以将while循环保留在主外壳程序中，而不是子外壳程序中.因此，您在while循环中创建的变量将在循环退出后保留其值.

Using process substitution like this allows you to keep your while loop in the main shell, not a subshell. Because of this, variables that you create in the while loop will retain their value after the loop exits.

显示的代码将打印CSV文件的第二列.如果这是代码应做的全部工作，则可以将其替换为:

The code as shown prints the second column of a CSV file. If that is all that the code is supposed to do, then it can be replaced with:

awk -F, '{print $2}' /some_directory/directory/file_in_question.csv

这篇关于带尾-n读行时如何使用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！