从重定向到日志文件的程序输出中删除进度条 [英] Removing progress bar from program output redirected into log file

问题描述

我正在运行一个程序，它将输出进度条.我是这样做的

I was running a program, and it will output the progress bar. I did it like this

python train.py |& tee train.log

train.log如下所示.

这是第1行

Training ...

这是第2行

...
[000] valid: 100%|█████████████████████████████████████████████████████████████▉| 2630/2631 [15:24<00:00,  2.98 track/s]
[000] valid: 100%|██████████████████████████████████████████████████████████████| 2631/2631 [15:25<00:00,  3.02 track/s]                                                                                                              
Epoch 000: train=0.11940351 valid=0.10640465 best=0.1064 duration=0.79 days

这是第3行 ...

This is line 3 ...

[001] valid: 100%|█████████████████████████████████████████████████████████████▉| 2629/2631 [15:11<00:00,  2.90 
[001] valid: 100%|█████████████████████████████████████████████████████████████▉| 2630/2631 [15:11<00:00,  2.89 
[001] valid: 100%|██████████████████████████████████████████████████████████████| 2631/2631 [15:12<00:00,  2.88                                                                                                   
Epoch 001: train=0.10971066 valid=0.09931737 best=0.0993 duration=0.79 days

在终端上，应该将它们视为替换自身，因此，在日志文件中，存在很多重复.因此，当我执行wc -l train.log时，它仅返回3行.但是，当我在文本编辑器中打开此5MB文本文件时，大约有20000行.

On the terminal, they are supposed to be viewed as replacing itself, hence in the log file, there are alot of repetitions. So when I did wc -l train.log, it only returned 3 lines. However when I opened this 5MB text file in the text editor, there are like 20000 lines.

我的目标是仅获取以下详细信息:

My objective is to only get these details:

Epoch 000: train=0.11940351 valid=0.10640465 best=0.1064 duration=0.79 days    
Epoch 001: train=0.10971066 valid=0.09931737 best=0.0993 duration=0.79 days

我的问题是:

1. 如何在不停止当前培训进度的情况下，从train.log的假定"3"行中提取所需的详细信息?请记住，此培训将持续进行10个以上的时间，因此我不想在编辑器中打开整个进度条.

1. How do I, without stopping my current training progress, extract my desired details from the suppposedly "3" lines of train.log? Keep in mind that this training will be continuously done for 10 more epochs, so I don't want to open the whole junk of progress bar in the editor.

将来，我应该如何存储日志文件(而不是调用python train.py |& tee train.log)，以便在终端中可以看到进度条时，仅将重要信息保留在文本文件中? /p>

In the future, how should I store my log file (instead of calling python train.py |& tee train.log) such that while I can see the progress bar in the terminal, I only keep the important information in a text file?

这是文件 train.log

推荐答案

进度条可能已写入stderr，您可以使用|&将进度条与stdout一起发送到tee.

The progress bars are probably written to stderr, which you send to tee together with stdout by using |&.

要仅将标准输出写入文件，请使用常规管道|.

To write only stdout to the file, use the normal pipe | instead.

进度条是通过写一行然后生成回车符(\r)而没有换行符字符(\n)生成的.要解决此问题并能够进一步处理文件，可以使用例如sed 's/\r/\n/g'.

The progress bar was generated by writing one line and then a carriage return character (\r) but no newline character (\n). To fix that and to be able to process the file further, you can use for example sed 's/\r/\n/g'.

以下内容适用于问题中链接的文件:

The following works with the file linked in the question:

$ sed 's/\r/\n/g' train.log | grep Epoch
Epoch 000: train=0.11940351 valid=0.10640465 best=0.1064 duration=0.79 days

这篇关于从重定向到日志文件的程序输出中删除进度条的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！