Condor 输出文件更新 [英] Condor output file updating

问题描述

我正在使用 Condor 运行多个模拟，并对程序进行编码，以便它在控制台中输出进度状态.这是在循环结束时完成的，它只是打印当前时间(这也可以是百分比或已用时间).代码如下所示:

<前>printf("开始");而(programNeedsToRum){//运行代码重复代码...//打印程序状态更新printf("[%i:%i:%i]\r\n", 小时、分钟、秒);}printf("完成");

当正常执行时(即在终端/cmd/bash 中)这工作正常，但秃鹰节点似乎没有 printf() 状态.只有在模拟完成后，所有的状态更新才被输出到文件中，但之后就不再使用了.我提交给 condor 的 *.sub 文件如下所示:

<前>宇宙 = 香草可执行文件 = 程序输出=输出/输出-$(进程)错误 = out/err-$(Process)排队 100

提交后，程序执行(这在 condor_q 中得到确认)并且输出文件包含以下内容:

<前>开始

只有在程序运行完毕后，其相应的输出文件才会显示(示例):

<前>开始[0:3:4][0:8:13][0:12:57][0:18:44]结束

当程序执行时，输出文件只包含START 文本.所以我得出的结论是，如果节点执行程序忙，则文件不会更新.所以我的问题是，有没有办法手动更新输出文件或以更好的方式收集有关程序进度的任何信息?

已经谢谢了

最大

解决方案

您想要做的是使用流输出选项.请参阅您可以传递给 condor_submit 的 stream_error 和 stream_output 选项，如下所述:http://research.cs.wisc.edu/htcondor/manual/current/condor_submit.html

默认情况下，HTCondor 将 stdout 和 stderr 本地存储在执行节点上，并在作业完成时将它们传输回提交节点.将 stream_output 设置为 TRUE 将要求 HTCondor 在输出发生时将其流式传输回提交节点.然后，您可以在它发生时对其进行检查.

I'm running several simulations using Condor and have coded the program so that it outputs a progress status in the console. This is done at the end of a loop where it simply prints the current time (this can also be percentage or elapsed time). The code looks something like this:

printf("START");
while (programNeedsToRum) {

   // Run code repetitive code...

   // Print program status update
   printf("[%i:%i:%i]\r\n", hours, minutes, seconds);
}
printf("FINISH");

When executing normally (i.e. in the terminal/cmd/bash) this works fine, but the condor nodes don't seem to printf() the status. Only once the simulation has finished, all the status updates have been outputted to the file but then it's no longer of use. My *.sub file that I submit to condor looks like this:

universe = vanilla
executable = program
output = out/out-$(Process)
error = out/err-$(Process)
queue 100

When submitted the program executes (this is confirmed in condor_q) and the output files contain this:

START

Only once the program has finished running its corresponding output file shows (example):

START
[0:3:4]
[0:8:13]
[0:12:57]
[0:18:44]
FINISH

Whilst the program executes, the output file only contains the START text. So I came to the conclusion that the file is not updated if the node executing program is busy. So my question is, is there a way of updating the output files manually or gather any information on the program's progress in a better way?

Thanks already

Max

解决方案

What you want to do is use the streaming output options. See the stream_error and stream_output options you can pass to condor_submit as outlined here: http://research.cs.wisc.edu/htcondor/manual/current/condor_submit.html

By default, HTCondor stores stdout and stderr locally on the execute node and transfers them back to the submit node on job completion. Setting stream_output to TRUE will ask HTCondor to instead stream the output as it occurs back to the submit node. You can then inspect it as it happens.

这篇关于Condor 输出文件更新的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！