如果失败的测试使连续构建失败？ [英] Should failing tests make the continuous build fail?
问题描述
如果有一个项目有在构建机器上作为构建过程的一部分执行的测试，如果设置的测试失败，那么整个构建是否应该失败？
什么是应该考虑什么时候回答这个问题？
If one has a project that has tests that are executed as part of the build procedure on a build machine, if a set tests fail, should the entire build fail?
What are the things one should consider when answering that question? Does it matter which tests are failing?
提示此问题的背景信息：
Background information that prompted this question:
目前，我正在开发一个具有 NUnit 测试的项目，这些测试是构建过程，并在我们的巡航控制.net 构建机器上执行。
Currently I am working on a project that has NUnit tests that are done as part of the build procedure and are executed on our cruise control .net build machine.
这个项目用于设置，如果任何测试失败，构建失败。推理是，如果测试失败，那意味着产品不工作/不完整/它是项目的失败，因此构建应该失败。
The project used to be setup so that if any tests fail, the build fails. The reasoning being if the tests fail, that means the product is not working/not complete/it is a failure of a project, and hence the build should fail.
我们添加了一些测试，虽然他们失败，它们对项目不是至关重要（见下面的更多细节）。因此，如果这些测试失败，该项目不是一个完全失败，我们仍然希望它构建。
We have added some tests that although they fail, they are not crucial to the project (see below for more details). So if those tests fail, the project is not a complete failure, and we would still want it to build.
其中一个测试通过验证不正确的参数导致一个异常，但测试不通过是检查所有允许的参数不导致异常的那个。所以类拒绝所有无效的情况，但也有一些有效的。这不是项目的问题，因为被拒绝的有效参数是应用程序不会依赖的边缘情况。
One of the tests that passes verifies that incorrect arguments result in an exception, but the test does not pass is the one that checks that all the allowed arguments do not result in an exception. So the class rejects all invalid cases, but also some valid ones. This is not a problem for the project, since the rejected valid arguments are fringe cases, on which the application will not rely.
推荐答案
如果它是以任何方式可行，那么做。它大大减少了破窗的问题：
If it's in any way doable, then do it. It greatly reduces the broken-window-problem:
在没有（可见）缺陷的系统中，引入一个小缺陷通常被视为一个非常糟糕的主意。所以如果你有一个绿色状态的项目（没有单元测试失败），并介绍第一个失败的测试，那么你（和/或你的同行）将有动机解决这个问题。
In a system with no (visible) flaws, introducing a small flaw is usually seen as a very bad idea. So if you've got a project with a green status (no unit test fails) and you introduce the first failing test, then you (and/or your peers) will be motivated to fix the problem.
如果在另一方面有已知的失败测试，那么只添加另一个破坏的测试就可以保持状态。
If, on the other side, there are known-failing tests, then adding just another broken test is seen as keeping the status-quo.
因此，您应该始终努力保持所有测试运行（而不仅仅是大多数测试）。将每一个失败的测试作为失败的原因，都会对实现这一目标有很大的帮助。
Therefore you should always strive to keep all tests running (and not just "most of them"). And treating every single failing test as reason for failing the build goes a long way towards that goal.
这篇关于如果失败的测试使连续构建失败？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
