Bash脚本超级慢 [英] Bash Script is super slow

问题描述

我正在更新一个旧脚本以解析ARP数据并从中获取有用的信息.我们添加了一个新的路由器，虽然我可以将ARP数据从路由器中提取出来，但是它是一种新格式.我有一个文件"zTempMonth"，它是来自两套路由器的所有arp数据，我需要将其编译为已标准化的新数据格式.下面的代码行完成了我需要它们逻辑上的工作-但速度非常慢-因为运行这些循环需要几天的时间，而以前的脚本要花20-30分钟.有没有一种方法可以加快它的速度，或者确定正在减慢它的速度?

I'm updating an old script to parse ARP data and get useful information out of it. We added a new router and while I can pull the ARP data out of the router it's in a new format. I've got a file "zTempMonth" which is a all the arp data from both sets of routers that I need to compile down into a new data format that's normalized. The below lines of code do what I need them to logically - but it's extremely slow - as in it will take days to run these loops where previously the script took 20-30 minutes. Is there a way to speed this up, or identify what's slowing it down?

先谢谢您

    echo "Parsing zTempMonth"
    while read LINE
    do
            wc=`echo $LINE | wc -w`
            if [[ $wc -eq "6" ]]; then
                    true
                    out=$(echo $LINE | awk '{ print $2 " " $4 " " $6}')
                    echo $out >> zTempMonth.tmp

            else
                    false
            fi

            if [[ $wc -eq "4" ]]; then
                    true
                    out=$(echo $LINE | awk '{ print $1 " " $3 " " $4}')
                    echo $out >> zTempMonth.tmp
            else
                    false
            fi


    done < zTempMonth

推荐答案

1. 虽然读取循环很慢.
2. 循环中的子外壳很慢.
循环中的
3. >>(open(f, 'a'))调用很慢.

1. While read loops are slow.
2. Subshells in a loop are slow.
3. >> (open(f, 'a')) calls in a loop are slow.

您只需加快#2和#3的速度即可加快速度并保持纯扑扑状态:

You could speed this up and remain in pure bash, just by losing #2 and #3:

#!/usr/bin/env bash

while read -a line; do
    case "${#line[@]}" in
        6) printf '%s %s %s\n' "${line[1]}" "${line[3]}" "${line[5]}";;
        4) printf '%s %s %s\n' "${line[0]}" "${line[2]}" "${line[3]}";;
    esac
done < zTempMonth >> zTempMonth.tmp

但是，如果有多行，这仍然会比纯awk慢.考虑一下这样简单的awk脚本:

But if there are more than a few lines, this will still be slower than pure awk. Consider an awk script as simple as this:

BEGIN {
    print "Parsing zTempMonth"
}   

NF == 6 {
    print $2 " " $4 " " $6
}   

NF == 4 {
    print $1 " " $3 " " $4
}   

您可以这样执行它:

awk -f thatAwkScript zTempMonth >> zTempMonth.tmp

获得与当前脚本相同的追加方法.

to get the same append approach as your current script.

这篇关于Bash脚本超级慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！