while循环读取文件非常慢 [英] while loop extremely slow read file

问题描述

我有一个while循环，该循环读取ftp日志文件并将其放入数组中，这样我就可以在数组中搜索并匹配/搜索流.不幸的是，while循环要花很长时间才能遍历文件，它是一个很大的文件，但是必须有另一种更快的方式来完成此操作.

I have a while loop that that reads in a ftp log file and puts it into an array so I'll be able to search through the array and match up/search for a flow. Unfortunately the while loop is taking forever to get through the file, it is a very large file but there must be another faster way of doing this.

# read file into array for original search results
while read FTP_SEARCH
do
ogl_date[count]=`echo $FTP_SEARCH | awk '{print $1, $2}'`
ogl_time[count]=`echo $FTP_SEARCH | awk '{print $3}'`
ogl_server[count]=`echo $FTP_SEARCH | awk '{print $4}'`
ogl_id[count]=`echo $FTP_SEARCH | awk '{print $5}'`
ogl_type[count]=`echo $FTP_SEARCH | awk -F '[' '{print $1}' | awk '{print $5}'`
ogl_pid[count]=`echo $FTP_SEARCH | awk -F'[' '{print $2}' | awk -F']' '{print $1}'`
ogl_commands[count]=`echo $FTP_SEARCH | awk '{
    for(i = 6; i <= NF; i++) 
        print $i;
    }'`

let "count += 1"

done < /tmp/ftp_search.14-12-02


Dec  1 23:59:03 sslmftp1 ftpd[4152]: USER xxxxxx  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: PASS password  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: FTP LOGIN FROM 172.19.x.xx [172.19.x.xx], xxxxxx  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: PWD  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: CWD /test/data/872507/  
Dec  1 23:59:03 sslmftp1 ftpd[4152]: TYPE Image`
Dec  1 23:59:03 sslmftp1 ftpd[4152]: PASV
Dec  1 23:59:04 sslmftp1 ftpd[4152]: NLST
Dec  1 23:59:04 sslmftp1 ftpd[4152]: FTP session closed
Dec  1 23:59:05 sslmftp1 ftpd[4683]: USER xxxxxx 
Dec  1 23:59:05 sslmftp1 ftpd[4683]: PASS password
Dec  1 23:59:05 sslmftp1 ftpd[4683]: FTP LOGIN FROM 172.19.1.24 [172.19.x.xx], xxxxxx 
Dec  1 23:59:05 sslmftp1 ftpd[4683]: PWD
Dec  1 23:59:05 sslmftp1 ftpd[4683]: CWD /test/data/944837/
Dec  1 23:59:05 sslmftp1 ftpd[4683]: TYPE Image

推荐答案

• 您无需保留迭代器即可添加到数组.您可以简单地执行array+=(item)(不 array+=item).
• 在输入中获取列就像使用read和多个目标变量一样简单.另外，最后一个变量获得第N个单词和所有后续单词.参见help [r]ead.
• You don't need to keep an iterator to add to arrays. You can simply do array+=(item) (not array+=item).
• Getting the columns in the input is as simple as using read with multiple target variables. As a bonus, the last variable gets the Nth word and all subsequent words. See help [r]ead.

这样可以节省大量的叉子，但是我还没有测试过它的速度.

This saves a ton of forks, but I haven't tested how fast it is.

ogl_date=()
[...]
ogl_commands=()

while read -r date1 date2 time server id type pid commands
do
    ogl_date+=("$date1 $date2")
    [...]
    ogl_commands+=("$commands")
done < /tmp/ftp_search.14-12-02

这篇关于while循环读取文件非常慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！