如何优化脚本着色日志文件 [英] How to optimize script for colorizing log files
本文介绍了如何优化脚本着色日志文件的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
下面的脚本着色的埋在日志文件中的SQL命令，和其他几个标签：
红=\\ X1B [31米
绿色=\\ X1B [32米
黄=\\ X1B [33米
蓝=\\ X1B [34米
白=\\ X1B [37米BLACK =\\ X1B [30; 100万
RED =\\ X1B [31; 100万
绿色=\\ X1B [32; 100万
黄色=\\ X1B [33; 100万
蓝色=\\ X1B [34; 100万
青色=\\ X1B [36; 100万
WHITE =\\ X1B [37; 100万onred =\\ X1B [41米lblack =\\ X1B [90米
lred =\\ X1B [91米
lgreen =\\ X1B [92米
lyellow =\\ X1B [93米
lblue =\\ X1B [94米
lmagenta =\\ X1B [95米
lcyan =\\ X1B [96米
lwhite =\\ X1B [97米reset_color =\\ X1B [0米SED -rS /'[^'] *'/ $ {青色}＆放大器; $ {} reset_color /克;
S / [A-Z _] * _ ID / $ {白}＆放大器; $ {} reset_color /克; s/(.*\\[)(AbstractApplicationContext)(\\].*)/${BLACK}\\\\1${reset_color}${yellow}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(ActionService)(\\].*)/${BLACK}\\\\1${reset_color}${lyellow}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(Authenticated)(\\].*)/${BLACK}\\\\1${reset_color}${lyellow}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(CascadeHandlerImpl)(\\].*)/${BLACK}\\\\1${reset_color}${lred}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(ClasspathHacker)(\\].*)/${BLACK}\\\\1${reset_color}${lred}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(ConfigManagerImpl|ConfigManagerLoader)(\\].*)/${BLACK}\\\\1${reset_color}${lred}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(ContextFilter|ContextImpl|ContextLoader)(\\].*)/${BLACK}\\\\1${reset_color}${lred}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(DataSourceRestrictionConverter)(\\].*)/${BLACK}\\\\1${reset_color}${lgreen}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(DatabaseLoader)(\\].*)/${BLACK}\\\\1${reset_color}${lgreen}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(DefaultListableBeanFactory)(\\].*)/${BLACK}\\\\1${reset_color}${lgreen}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(DispatchFilter)(\\].*)/${BLACK}\\\\1${reset_color}${lgreen}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(FileHelper|FileIndex)(\\].*)/${BLACK}\\\\1${reset_color}${lyellow}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(LicenseManagerImpl)(\\].*)/${BLACK}\\\\1${reset_color}${lblue}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(LocalizedStringsLoader)(\\].*)/${BLACK}\\\\1${reset_color}${lblue}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(LoggingPropertyPlaceholderConfigurer)(\\].*)/${BLACK}\\\\1${reset_color}${lblue}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(PooledDbDriverImpl)(\\].*)/${BLACK}\\\\1${reset_color}${lcyan}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(ProjectLoader)(\\].*)/${BLACK}\\\\1${reset_color}${lcyan}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(PropertiesLoaderSupport)(\\].*)/${BLACK}\\\\1${reset_color}${lcyan}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(RenderingFilter)(\\].*)/${BLACK}\\\\1${reset_color}${lwhite}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(SecurityControllerImpl|SecurityServiceImpl)(\\].*)/${BLACK}\\\\1${reset_color}${lblue}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(WorkflowRulesContainerImpl|WorkflowRulesContainerLoader)(\\].*)/${BLACK}\\\\1${reset_color}${lred}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g;
s/(.*\\[)(XmlBeanDefinitionReader)(\\].*)/${BLACK}\\\\1${reset_color}${lgreen}\\\\2${reset_color}${BLACK}\\\\3${reset_color}/g; S /（选择|选择）（。*）（（FROM |从）（[^）] *））（[^ \\] *）（其中|？在哪里）/ $ {}黄1 \\\\ $ {reset_color } \\\\ 2 $ {}黄\\\\ 4 $ {reset_color} $ {}黄色5 \\\\ $ {} reset_color 6 \\\\ $ {}黄色7 \\\\ $ {} reset_color /克;
S / LEFT OUTER JOIN [^] * ON / $ {}黄色和放大器; $ {} reset_color /克;
S / ORDER BY / $ {}黄色和放大器; $ {} reset_color /克;
S /（ASC | DESC）/ $ {}黄色和放大器; $ {} reset_color /克;
S / GROUP BY / $ {}黄色和放大器; $ {} reset_color /克;
S /（INSERT INTO）（[^] *）（*）（值）/ $ {}绿1 \\\\ $ {reset_color} $ {}绿色\\\\ 2 $ {} reset_color 3 \\\\ $ {绿} \\ \\ 4 $ {} reset_color /克;
S /（INSERT INTO）（[^] *）/ $ {}绿1 \\\\ $ {reset_color} $ {}绿色\\\\ 2 $ {} reset_color 3 \\\\ $ {} reset_color /克（*）;
S /（更新）*（[^] *）（SET |设置）/ $ {}蓝1 \\\\ $ {reset_color} $ {} BLUE \\\\ 2 $ {reset_color} $ {}蓝3 \\\\ $ {reset_color }/G;
S / DELETE FROM * [^] * WHERE / $ {RED}＆放大器; $ {} reset_color /克; S / \\ * \\ * \\ * ROLLBACK \\ * \\ * \\ * / $ {白} $ {onred}＆放大器; $ {} reset_color /克;
S / \\ [错误\\] / $ {WHITE} $ {onred}＆放大器; $ {} reset_color /克;
S / SQLServerException：/ $ {WHITE} $ {onred}＆放大器; $ {} reset_color / g的
虽然，当在1.9 MB的日志文件来运行，它需要1点41（1分钟41秒），以执行...而简单的猫在小于0时01运行...
同样的结果（在1秒钟）如果我删除AbstractApplicationContext和XmlBeanDefinitionReader。
之间的块我不明白为什么那个特定的块使这样的差别！？
有没有办法来优化这种彩色化脚本？
示例文件提取物（复制为使其成为一个大的文件）：
[INFO] [2016年5月20日16：17：51346] [的ContextLoader] - [根的WebApplicationContext：初始化开始]
[INFO] [2016年5月20日16：17：51505] [XmlBeanDefinitionReader] - [从ServletContext的资源加载XML bean定义[/WEB-INF/config/context/appContext.xml]
[INFO] [2016年5月20日16：17：52986] [PropertiesLoaderSupport] - [加载特性从类路径中的资源文件[配置/ mail.properties]
[INFO] [2016年5月20日16：17：55900] [ConfigManagerLoader] - [读取XML配置]
[INFO] [2016年5月20日16：17：55991] [ConfigManagerLoader] - [读取XML配置：确定]
[WARN] [二零一六年五月二十零日16：17：56384] [ConfigManagerLoader] - [低最大内存= 477102080。 Java的最大内存= 1000 MB，建议在生产中使用，作为最低限度。]
[INFO] [2016年5月20日16：17：58309] [LocalizedStringsLoader] - [区域设置加载本地化字符串= [fr_FR目录]
[INFO] [2016年5月20日16：17：58337] [LocalizedStringsLoader] - [区域设置加载本地化字符串= [fr_FR目录]：OK，字符串：759]
[INFO] [2016年5月20日16：17：58641] [LocalizedStringsLoader] - [区域设置加载本地化字符串= [fr_FR目录]
[INFO] [2016年5月20日16：17：58768] [LocalizedStringsLoader] - [区域设置加载本地化字符串= [fr_FR目录]：OK，字符串：46436]
[INFO] [2016年5月20日16：17：58830] [LocalizedStringsLoader] - [区域设置加载本地化字符串= [nl_NL]
[INFO] [2016年5月20日16：17：58946] [LocalizedStringsLoader] - [区域设置加载本地化字符串= [nl_NL]：OK，字符串：46436]
[INFO] [2016年5月20日16：17：59434] [PropertiesLoaderSupport] - [加载特性从类路径中的资源文件[配置/ mail.properties]
[INFO] [2016年5月20日16：18：00476] [XmlBeanDefinitionReader] - [从类路径资源加载XML bean定义[项目儿童的context.xml]
[DEBUG] [二零一六年五月二十日16：18：01259] [DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL关]
[DEBUG] [二零一六年五月二十日16：18：01340] [DbConnectionImpl] - [更新时间：-1记录]
[INFO] [2016年5月20日16：18：01363] [DatabaseImpl] - [数据库加载：数据]
[INFO] [2016年5月20日16：18：01379] [DatabaseLoader] - [装载数据库[数据]：确定]
[INFO] [2016年5月20日16：18：01393] [DatabaseLoader] - [装载数据库[方案]
[DEBUG] [二零一六年五月二十日16：18：01865] [DbConnectionImpl] - [SELECT FROM表列在表格名='样品']
[DEBUG] [二零一六年五月二十日16：18：01894] [DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL关]
[DEBUG] [二零一六年五月二十日16：18：01898] [DbConnectionImpl] - [更新时间：-1记录]
[INFO] [2016年5月20日16：18：06241] [WorkflowRulesContainerLoader] - [导入工作流规则，ruleId = [checkRequestDuplicates]
[INFO] [2016年5月20日16：18：06384] [WorkflowRulesContainerLoader] - [导入工作流规则，ruleId = [getStatistic]
[INFO] [2016年5月20日16：18：06971] [WorkflowRulesContainerLoader] - [导入工作流规则，ruleId = [saveRecord]
[INFO] [2016年5月20日16：18：07126] [WorkflowRulesContainerLoader] - [导入工作流规则，ruleId = [EmailService]
[INFO] [2016年5月20日16：18：07542] [WorkflowRulesContainerLoader] - [导入工作流规则，ruleId = [LocalizationRead]
[INFO] [2016年5月20日16：18：09578] [FileIndex $ 1] - [文件装载指数开始]
[DEBUG] [二零一六年五月二十日16：18：19406] [DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL关]
[DEBUG] [二零一六年五月二十日16：18：19410] [DbConnectionImpl] - [更新时间：-1记录]
[INFO] [2016年5月20日16：18：22201] [LicenseManagerImpl] - [结帐并发许可= [主]
[DEBUG] [二零一六年五月二十日16：18：22209] [SecurityControllerImpl] - [确定下一个请求]
[DEBUG] [二零一六年五月二十日16：18：22239] [RenderingFilter] - [渲染模式]
[DEBUG] [二零一六年五月二十日16：18：22253] [认证] - [用户进行身份验证]
[DEBUG] [二零一六年五月二十日16：18：22306] [FileHelper] - [查找文件= [hovertip.js]
[DEBUG] [二零一六年五月二十日16：18：22399] [FileHelper] - [查找文件= [hovertip.css]
[DEBUG] [16二○一六年五月二十〇日：22：18263] [DbConnectionImpl] - [INSERT INTO notifications_log（列，列）VALUES（'二零一六年五月二十〇日'，'错误'）]
[DEBUG] [16二○一六年五月二十日：22：18,334] [DbConnectionImpl] - [更新时间：1条记录]
[DEBUG] [16二○一六年五月二十日：22：18393] [DbConnectionImpl] - [*** *** COMMIT]
[DEBUG] [16二○一六年五月二十日：22：18549] [DbConnectionImpl] - [*** *** ROLLBACK]
[DEBUG] [16二○一六年五月二十日：23：37659] [DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL关]
[DEBUG] [16二○一六年五月二十○日：23：37662] [DbConnectionImpl] - [更新时间：-1记录]
[DEBUG] [16二○一六年五月二十日：23：37886] [在DataSourceImpl] - [SELECT col_id FROM TABLE_1 LEFT OUTER JOIN TABLE_2 ON table_1.col_id = table_2.col_id]
[DEBUG] [16二○一六年五月二十日：23：37926] [DbConnectionImpl] - [*** *** COMMIT]
[DEBUG] [16二○一六年五月二十日：23：37930] [ContextFilter] - [----------要求：加工成品]
[DEBUG] [16二○一六年五月二十○日：38：38,033] [DbConnectionImpl] - [UPDATE设置用户PWD = NULL其中用户名='我']
[DEBUG] [16二○一六年五月二十○日：38：38051] [DbConnectionImpl] - [更新时间：1条记录]
[DEBUG] [16二○一六年五月二十○日：38：38058] [DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL关]
[DEBUG] [16二○一六年五月二十日：38：38063] [DbConnectionImpl] - [更新时间：-1记录]
[DEBUG] [17二○一六年五月二十日：43：25087] [DbConnectionImpl] - [*** *** COMMIT]
[DEBUG] [17二○一六年五月二十日：43：25096] [ContextFilter] - [----------要求：加工成品]
解决方案
它采取了很长的时间，因为你每一个输入行比较对您37正则表达式的每一位！用awk，只是做了测试一次，例如：
$猫文件
[INFO] [TS 1] [的ContextLoader] - [休息1]
[INFO] [TS 2] [XmlBeanDefinitionReader] - [休息2]
[INFO] [TS 3] [PropertiesLoaderSupport] - [休息3]
$
$猫tst.awk
开始 {
红色=＆LT;红＆gt;中＃\\ X1B [31米
绿色=＆LT;绿色＆gt;中＃\\ X1B [32米
黄=＆LT;黄色＆gt;中＃\\ X1B [33米
黑=＆LT;黑色＆gt;中＃\\ X1B [30; 100万
复位=＆LT;被重置GT; ＃\\ X1B [0米 颜色[的ContextLoader] =红
颜色[XmlBeanDefinitionReader] =绿色
颜色[PropertiesLoaderSupport] =黄色
}
匹配（$ 0 /（（\\ [[^] + \\]）{2} \\ []）（[^] +）（。*）/，一）{
打印黑色[1]复位颜色[A [3]一[3]复位黑色[4]复位
}$ AWK -f tst.awk文件
＆LT;黑＆GT; [INFO] [TS 1] [＆LT;复位＆GT;＆LT;红＆GT;的ContextLoader＆LT;复位＆GT;＆LT;黑＆GT;] - [休息1];复位＆GT;
＆LT;黑＆GT; [INFO] [TS 2] [＆LT;复位＆GT;＆LT;绿色＆GT; XmlBeanDefinitionReader＆LT;复位＆GT;＆LT;黑＆GT;] - [其余2]＆LT;复位＆GT;
＆LT;黑＆GT; [INFO] [TS 3] [＆LT;复位＆GT;＆LT;黄色＆GT; PropertiesLoaderSupport＆LT;复位＆GT;＆LT;黑＆GT;] - [其余3＆LT;复位＆GT;
以上菌株的任何字符串出现在每一个你关心线的部分，然后查找该字符串的打印散列的颜色。它使用了GNU AWK的第三个参数匹配（），它是其他awks一个简单的好办法。
我使用的颜色的名称，如＆LT;红＆GT; 只是让你可以看到输出。在真实系统你只是写红=\\ X1B [31米而不是红=＆LT;红＆gt;中＃\\ X1B [31米等。
要更新上述处理SQL语句也从sed脚本，模拟的逻辑：
S /（选择|选择）（*）（（FROM |从）（[^）] *））（[^ \\] *）（WHERE |哪里） ？/ $ {}黄1 \\\\ $ {} reset_color \\\\ 2 $ {}黄\\\\ 4 $ {reset_color} $ {}黄色5 \\\\ $ {} reset_color 6 \\\\ $ {}黄色7 \\\\ $ { reset_color} /克;
S / LEFT OUTER JOIN [^] * ON / $ {}黄色和放大器; $ {} reset_color /克;
将类似（从现有的awk脚本中这个答案首块）：
....
匹配（$ 0 /（（\\ [[^] + \\]）{2} \\ []）（[^] +）（。*）/，一）{
打印黑色[1]复位颜色[A [3]一[3]复位黑色[4]复位
下一个
}
比赛（$ 0 /（选择|选择）（*）（（FROM |从）（[^）] *））（[^ \\] *）（WHERE |哪里）？/，A）{
打印黄色[1]复位[2]黄色[4]复位黄色[5]重置[6]黄复位
下一个
}
比赛（$ 0 / LEFT OUTER JOIN [^] * ON /，A）{
打印黄色[0]复位
下一个
}
....
请注意在每块的末尾下一步言 - 告诉awk将停止处理输入的当前行，回到开始它的隐含同时读取行工作循环。我们使用的效率，从分析行后再次，它已经成功地对线路匹配一个正则表达式停止awk的。
The following script colorizes SQL commands buried in log files, and a couple of other "tags":
red="\x1b[31m"
green="\x1b[32m"
yellow="\x1b[33m"
blue="\x1b[34m"
white="\x1b[37m"
BLACK="\x1b[30;1m"
RED="\x1b[31;1m"
GREEN="\x1b[32;1m"
YELLOW="\x1b[33;1m"
BLUE="\x1b[34;1m"
CYAN="\x1b[36;1m"
WHITE="\x1b[37;1m"
onred="\x1b[41m"
lblack="\x1b[90m"
lred="\x1b[91m"
lgreen="\x1b[92m"
lyellow="\x1b[93m"
lblue="\x1b[94m"
lmagenta="\x1b[95m"
lcyan="\x1b[96m"
lwhite="\x1b[97m"
reset_color="\x1b[0m"
sed -r "s/'[^']*'/${CYAN}&${reset_color}/g;
s/[a-z_]*_id/${white}&${reset_color}/g;
s/(.*\[)(AbstractApplicationContext)(\].*)/${BLACK}\\1${reset_color}${yellow}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(ActionService)(\].*)/${BLACK}\\1${reset_color}${lyellow}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(Authenticated)(\].*)/${BLACK}\\1${reset_color}${lyellow}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(CascadeHandlerImpl)(\].*)/${BLACK}\\1${reset_color}${lred}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(ClasspathHacker)(\].*)/${BLACK}\\1${reset_color}${lred}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(ConfigManagerImpl|ConfigManagerLoader)(\].*)/${BLACK}\\1${reset_color}${lred}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(ContextFilter|ContextImpl|ContextLoader)(\].*)/${BLACK}\\1${reset_color}${lred}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(DataSourceRestrictionConverter)(\].*)/${BLACK}\\1${reset_color}${lgreen}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(DatabaseLoader)(\].*)/${BLACK}\\1${reset_color}${lgreen}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(DefaultListableBeanFactory)(\].*)/${BLACK}\\1${reset_color}${lgreen}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(DispatchFilter)(\].*)/${BLACK}\\1${reset_color}${lgreen}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(FileHelper|FileIndex)(\].*)/${BLACK}\\1${reset_color}${lyellow}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(LicenseManagerImpl)(\].*)/${BLACK}\\1${reset_color}${lblue}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(LocalizedStringsLoader)(\].*)/${BLACK}\\1${reset_color}${lblue}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(LoggingPropertyPlaceholderConfigurer)(\].*)/${BLACK}\\1${reset_color}${lblue}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(PooledDbDriverImpl)(\].*)/${BLACK}\\1${reset_color}${lcyan}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(ProjectLoader)(\].*)/${BLACK}\\1${reset_color}${lcyan}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(PropertiesLoaderSupport)(\].*)/${BLACK}\\1${reset_color}${lcyan}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(RenderingFilter)(\].*)/${BLACK}\\1${reset_color}${lwhite}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(SecurityControllerImpl|SecurityServiceImpl)(\].*)/${BLACK}\\1${reset_color}${lblue}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(WorkflowRulesContainerImpl|WorkflowRulesContainerLoader)(\].*)/${BLACK}\\1${reset_color}${lred}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(.*\[)(XmlBeanDefinitionReader)(\].*)/${BLACK}\\1${reset_color}${lgreen}\\2${reset_color}${BLACK}\\3${reset_color}/g;
s/(SELECT|select)(.*)((FROM|from) ([^ )]*))([^\]]*)(WHERE|where)?/${yellow}\\1${reset_color}\\2${yellow}\\4${reset_color} ${YELLOW}\\5${reset_color}\\6${yellow}\\7${reset_color}/g;
s/LEFT OUTER JOIN [^ ]* ON/${yellow}&${reset_color}/g;
s/ORDER BY/${yellow}&${reset_color}/g;
s/(ASC|DESC)/${yellow}&${reset_color}/g;
s/GROUP BY/${yellow}&${reset_color}/g;
s/(INSERT INTO) ([^ ]*)(.*)(VALUES)/${green}\\1${reset_color} ${GREEN}\\2${reset_color}\\3${green}\\4${reset_color}/g;
s/(INSERT INTO) ([^ ]*)(.*)/${green}\\1${reset_color} ${GREEN}\\2${reset_color}\\3${reset_color}/g;
s/(UPDATE) *([^ ]*) (SET|set)/${blue}\\1${reset_color} ${BLUE}\\2${reset_color} ${blue}\\3${reset_color}/g;
s/DELETE FROM *[^ ]* WHERE/${RED}&${reset_color}/g;
s/\*\*\*ROLLBACK\*\*\*/${white}${onred}&${reset_color}/g;
s/\[ERROR\]/${WHITE}${onred}&${reset_color}/g;
s/SQLServerException:/${WHITE}${onred}&${reset_color}/g"
Though, when run on a 1.9 MB log file, it takes 1:41 (1 minute, 41 seconds) to execute... while a simple cat is run in less than 0:01...
Same results (under 1 sec) if I remove the block between "AbstractApplicationContext" and "XmlBeanDefinitionReader".
I don't understand why that particular block makes such a difference!?
Is there a way to optimize such colorizing script?
Sample file extract (to duplicate for making it become a large file):
[INFO ][2016-05-20 16:17:51,346][ContextLoader] - [Root WebApplicationContext: initialization started]
[INFO ][2016-05-20 16:17:51,505][XmlBeanDefinitionReader] - [Loading XML bean definitions from ServletContext resource [/WEB-INF/config/context/appContext.xml]]
[INFO ][2016-05-20 16:17:52,986][PropertiesLoaderSupport] - [Loading properties file from class path resource [config/mail.properties]]
[INFO ][2016-05-20 16:17:55,900][ConfigManagerLoader] - [Reading XML config]
[INFO ][2016-05-20 16:17:55,991][ConfigManagerLoader] - [Reading XML config: OK]
[WARN ][2016-05-20 16:17:56,384][ConfigManagerLoader] - [Low max memory=477102080. Java max memory=1000 MB is recommended for production use, as a minimum.]
[INFO ][2016-05-20 16:17:58,309][LocalizedStringsLoader] - [Loading localized strings for locale=[fr_FR]]
[INFO ][2016-05-20 16:17:58,337][LocalizedStringsLoader] - [Loading localized strings for locale=[fr_FR]: OK, strings:759]
[INFO ][2016-05-20 16:17:58,641][LocalizedStringsLoader] - [Loading localized strings for locale=[fr_FR]]
[INFO ][2016-05-20 16:17:58,768][LocalizedStringsLoader] - [Loading localized strings for locale=[fr_FR]: OK, strings:46436]
[INFO ][2016-05-20 16:17:58,830][LocalizedStringsLoader] - [Loading localized strings for locale=[nl_NL]]
[INFO ][2016-05-20 16:17:58,946][LocalizedStringsLoader] - [Loading localized strings for locale=[nl_NL]: OK, strings:46436]
[INFO ][2016-05-20 16:17:59,434][PropertiesLoaderSupport] - [Loading properties file from class path resource [config/mail.properties]]
[INFO ][2016-05-20 16:18:00,476][XmlBeanDefinitionReader] - [Loading XML bean definitions from class path resource [project-child-context.xml]]
[DEBUG][2016-05-20 16:18:01,259][DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL OFF]
[DEBUG][2016-05-20 16:18:01,340][DbConnectionImpl] - [Updated: -1 records]
[INFO ][2016-05-20 16:18:01,363][DatabaseImpl] - [Database loaded: data]
[INFO ][2016-05-20 16:18:01,379][DatabaseLoader] - [Loading Database [data]: OK]
[INFO ][2016-05-20 16:18:01,393][DatabaseLoader] - [Loading Database [schema]]
[DEBUG][2016-05-20 16:18:01,865][DbConnectionImpl] - [SELECT column FROM table WHERE table_name = 'sample']
[DEBUG][2016-05-20 16:18:01,894][DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL OFF]
[DEBUG][2016-05-20 16:18:01,898][DbConnectionImpl] - [Updated: -1 records]
[INFO ][2016-05-20 16:18:06,241][WorkflowRulesContainerLoader] - [Loading Workflow Rule, ruleId=[checkRequestDuplicates]]
[INFO ][2016-05-20 16:18:06,384][WorkflowRulesContainerLoader] - [Loading Workflow Rule, ruleId=[getStatistic]]
[INFO ][2016-05-20 16:18:06,971][WorkflowRulesContainerLoader] - [Loading Workflow Rule, ruleId=[saveRecord]]
[INFO ][2016-05-20 16:18:07,126][WorkflowRulesContainerLoader] - [Loading Workflow Rule, ruleId=[EmailService]]
[INFO ][2016-05-20 16:18:07,542][WorkflowRulesContainerLoader] - [Loading Workflow Rule, ruleId=[LocalizationRead]]
[INFO ][2016-05-20 16:18:09,578][FileIndex$1] - [File index loading started]
[DEBUG][2016-05-20 16:18:19,406][DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL OFF]
[DEBUG][2016-05-20 16:18:19,410][DbConnectionImpl] - [Updated: -1 records]
[INFO ][2016-05-20 16:18:22,201][LicenseManagerImpl] - [Checkout concurrent license=[Main]]
[DEBUG][2016-05-20 16:18:22,209][SecurityControllerImpl] - [Determine next request]
[DEBUG][2016-05-20 16:18:22,239][RenderingFilter] - [Rendering mode]
[DEBUG][2016-05-20 16:18:22,253][Authenticated] - [User is authenticated]
[DEBUG][2016-05-20 16:18:22,306][FileHelper] - [Find file=[hovertip.js]]
[DEBUG][2016-05-20 16:18:22,399][FileHelper] - [Find file=[hovertip.css]]
[DEBUG][2016-05-20 16:22:18,263][DbConnectionImpl] - [INSERT INTO notifications_log (columns, columns) VALUES ('2016-05-20', 'ERROR')]
[DEBUG][2016-05-20 16:22:18,334][DbConnectionImpl] - [Updated: 1 records]
[DEBUG][2016-05-20 16:22:18,393][DbConnectionImpl] - [***COMMIT***]
[DEBUG][2016-05-20 16:22:18,549][DbConnectionImpl] - [***ROLLBACK***]
[DEBUG][2016-05-20 16:23:37,659][DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL OFF]
[DEBUG][2016-05-20 16:23:37,662][DbConnectionImpl] - [Updated: -1 records]
[DEBUG][2016-05-20 16:23:37,886][DataSourceImpl] - [SELECT col_id FROM table_1 LEFT OUTER JOIN table_2 ON table_1.col_id=table_2.col_id]
[DEBUG][2016-05-20 16:23:37,926][DbConnectionImpl] - [***COMMIT***]
[DEBUG][2016-05-20 16:23:37,930][ContextFilter] - [---------- Request: processing finished]
[DEBUG][2016-05-20 16:38:38,033][DbConnectionImpl] - [UPDATE users SET pwd = NULL WHERE user_name = 'me']
[DEBUG][2016-05-20 16:38:38,051][DbConnectionImpl] - [Updated: 1 records]
[DEBUG][2016-05-20 16:38:38,058][DbConnectionImpl] - [SET CONCAT_NULL_YIELDS_NULL OFF]
[DEBUG][2016-05-20 16:38:38,063][DbConnectionImpl] - [Updated: -1 records]
[DEBUG][2016-05-20 17:43:25,087][DbConnectionImpl] - [***COMMIT***]
[DEBUG][2016-05-20 17:43:25,096][ContextFilter] - [---------- Request: processing finished]
解决方案
It's taking a very long time because you're comparing every input line against every one of your 37 regexps! Use awk and just do the test once, e.g.:
$ cat file
[INFO ][ts 1][ContextLoader] - [rest 1]
[INFO ][ts 2][XmlBeanDefinitionReader] - [rest 2]
[INFO ][ts 3][PropertiesLoaderSupport] - [rest 3]
$
$ cat tst.awk
BEGIN {
red = "<red>" # "\x1b[31m"
green = "<green>" # "\x1b[32m"
yellow = "<yellow>" # "\x1b[33m"
black = "<black>" # "\x1b[30;1m"
reset = "<reset>" # "\x1b[0m"
color["ContextLoader"] = red
color["XmlBeanDefinitionReader"] = green
color["PropertiesLoaderSupport"] = yellow
}
match($0,/((\[[^]]+\]){2}\[)([^]]+)(.*)/,a) {
print black a[1] reset color[a[3]] a[3] reset black a[4] reset
}
$ awk -f tst.awk file
<black>[INFO ][ts 1][<reset><red>ContextLoader<reset><black>] - [rest 1]<reset>
<black>[INFO ][ts 2][<reset><green>XmlBeanDefinitionReader<reset><black>] - [rest 2]<reset>
<black>[INFO ][ts 3][<reset><yellow>PropertiesLoaderSupport<reset><black>] - [rest 3]<reset>
The above isolates whatever string occurs in the section of each line you care about and then looks up the color of that string in a hash for printing. It uses GNU awk for the 3rd arg to match(), it's a simple tweak for other awks.
I used color names like <red> just so you could see the output. In your real system you'd just write red = "\x1b[31m" instead of red = "<red>" # "\x1b[31m" etc.
To update the above to handle the SQL statements too, emulating the logic from your sed script:
s/(SELECT|select)(.*)((FROM|from) ([^ )]*))([^\]]*)(WHERE|where)?/${yellow}\\1${reset_color}\\2${yellow}\\4${reset_color} ${YELLOW}\\5${reset_color}\\6${yellow}\\7${reset_color}/g;
s/LEFT OUTER JOIN [^ ]* ON/${yellow}&${reset_color}/g;
would be something like (first block from the existing awk script in this answer):
....
match($0,/((\[[^]]+\]){2}\[)([^]]+)(.*)/,a) {
print black a[1] reset color[a[3]] a[3] reset black a[4] reset
next
}
match($0,/(SELECT|select)(.*)((FROM|from) ([^ )]*))([^\]]*)(WHERE|where)?/,a) {
print yellow a[1] reset a[2] yellow a[4] reset yellow a[5] reset a[6] yellow reset
next
}
match($0,/LEFT OUTER JOIN [^ ]* ON/,a) {
print yellow a[0] reset
next
}
....
Note the "next" statement at the end of every block - that tells awk to stop processing the current line of input and go back to the start of it's implicit while read line work loop. We use that for efficiency to stop awk from analyzing the line again after it's already succeeded in matching a regexp against the line.
这篇关于如何优化脚本着色日志文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
