启用“指定批量大小"时，Azure DevOps 版本中缺少控制台输出 [英] Console output missing in Azure DevOps release when enabling "Specify a batch size"

问题描述

我们在 Azure DevOps 中运行了一个大型测试集(基于 NUnit).最近我们在 Visual Studio 测试任务中启用了重新运行失败的测试"选项.起初，由于 VSTest 中处理自定义测试显示名称(我们的测试所需)的错误，这不起作用.测试仍然只会运行一次.但是，通过设置

效果很好，除了一个奇怪的效果……没有明显的原因，详细的日志不再显示我们自己的自定义输出(由 Console.WriteLine 生成).这在以前没有问题.我们现在得到的只是默认输出和最终测试结果.

未启用指定批量大小"的日志提供更多信息:

日志本身仍然被写入:我们知道(几乎)所有信息也包含在每个测试中.这显示在测试结果中:

此外，只需禁用批量大小选项即可再次显示日志.

有谁知道是什么导致了这种行为，以及如何解决它?到目前为止，在 Console.Writeline/Trace.Writeline/Debug.Writeline 等之间切换并没有帮助，我还没有找到关于这个特定问题的很多其他信息......并且在某些情况下有一个完整的日志文件是必要的(或更实用的方式)，所以如果我们设法同时进行重试和完整日志记录会很好.

提前致谢！

解决方案

我发布了

在Notepad++中打开文件时，终于看到了我完整运行的日志:

- 所有这些行都是使用 Console.WriteLine() 编写的.

- 如果 trx 文件多于一个，则最大的文件包含完整运行的日志:小文件仅包含所选测​​试的日志.

注意:我们发现在测试运行超时期间，不会生成此文件.这已报告给 Microsoft 并被确认为一个问题:

明白了这一点.我们正在 vstest 任务中进行高级诊断.例如，如果测试需要更多时间来完成，我们将中止测试.在这种情况下，测试过程的转储将是也创建了，当然我们也会上传 trx.第二个场景，不是特定测试需要时间，但整体运行结束超时，我们将转储测试过程并中止运行.转储将帮助您调试问题."

在未来的某个地方，这个转储应该可以使用.

We have a large test set running (based on NUnit) within Azure DevOps. Recently we enabled the "Rerun failed tests" option within the Visual Studio Test task. At first, this didn't work due to a bug in VSTest with handling custom test display names (which are required for our tests). Tests would still only run once. However, by setting a batch size, this issue is fixed and tests are finally retried correctly.

It works wonderful, except for one strange effect... for no apparent reason, the detailed logs are no longer showing our own custom output (generated by Console.WriteLine). This worked without problems before. All we get now is default output and the final test result.

Logs without "Specify a batch size" enabled are way more informative:

The logs themselves are still written: we know that as (almost) all the information is also included per-test. This is shown in the test results:

Also, simply disabling the batch size option makes the logs show up again.

Does anyone have an idea what causes this behavior, and how to fix it? So far, switching around between Console.Writeline/Trace.Writeline/Debug.Writeline etc. hasn't helped, and I haven't found much other info about this specific issue... and there are certain situations where having one complete log file is necessary (or way more practical), so it would be nice if we manage to have both retries and full logging.

Thanks in advance!

解决方案

I posted the question on developercommunity.visualstudio.com as well. The discussion took a bit of time, but summarized, the following response was given:

"The supported behavior is to have the Trace information from the tests be present as part of the test results file (trx)/Standard console logs and not part of the build logs (as you have mentioned in your question). Do note this is also dependent on the test framework being used. There’s no plan to have the trace information flow to the build logs from the test. The different behavior you are seeing when you turn on Batching option is due to the fact the internal flow of execution changes a bit (and eventually all flows will converge to same as batch option in coming days). we recommend not to take dependency on build logs. Instead the test tab is the place you will get in better logs in context to test case/test run."

A full test run output should be available in the trx files found in the test run. Upon check, this is indeed the case:

When opening the file in Notepad++, I finally see the logs of my complete run:

- All these lines are written by using Console.WriteLine().

- If there are more trx files than one, then the largest file holds the logs of the complete run: the small file only has the logs for the selected test.

NOTE: We found that during a test run timeout, this file is not generated. This has been reported to Microsoft and is acknowledged as an issue:

"got the point here. we are working on Advanced diagnostics in vstest task. for instance we will abort the test if a test is taking more time to complete. in this case a dump of the test process will be created as well and ofcourse we will have trx also getting uploaded. the second scenario where not a perticular test is taking time but overall run ends time out we will take a dump of test process and abort the run. Dumps will help you debug the issue."

Somewhere in the future, this dump should be available to use.

这篇关于启用“指定批量大小"时，Azure DevOps 版本中缺少控制台输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！