查找未通过测试的提交，而无需在每次提交上都运行每个测试 [英] Find commit that broke a test without running every test on every commit

问题描述

我正在尝试找到一种可以解决以下问题的工具:

I am trying to find a tool that can solve the following problem:

我们的整个测试套件需要花费几个小时才能运行，这通常使得很难或至少非常耗时地找出哪个提交违反了特定的测试，因为在两次测试运行之间可能有50到200次提交.在任何给定时间，只有很少的破坏测试，因此与运行整个测试套件相比，仅重新运行破坏的测试非常快.

Our entire test suite takes hours to runs which often makes it difficult or at least very time consuming to find out which commit broke a specific test since there may be 50 to 200 commits in between test runs. At any given time there are only very few broken tests, so rerunning only the broken tests is very fast compared to running the entire test suite.

是否有工具，例如一个连续集成服务器，它可以在测试通过的最后一个修订版和测试通过的第一个修订版之间，通过几个修订版重新运行失败的测试，从而自动找出测试从哪个成功提交切换到哪个特定提交破碎的.

Is there a tool, e.g. a continuous integration server, that can rerun failed tests with a couple of revisions in between the last revision where the test was OK and the first revision where the test was not OK and therefore automatically find out on which specific commit a test switched from successful to broken.

例如:

在修订版100中，测试A和B可以.在修订版200中，测试A和B被破坏.

Test A and B are ok in revision 100. Test A and B are broken in revision 200.

该工具现在应在修订版150上运行两个测试. 那么如果测试A在测试版150中被破坏，测试B可以通过，它可以继续检查测试A的版本125和测试B的测试版本175，依此类推，直到可以通过特定的提交来解释每个测试失败.

The tool should now run both tests with revision 150. Then if e.g. test A was broken and test B was OK in revision 150, it could continue to check test A with revision 125 and test B with revision 175 and so on until every broken test can be explained by some specific commit.

对于一个测试，我可能会和git bisect一起破解一些东西.但是对于多次失败的测试，这可能还不够，因为我们需要在两个方向上搜索许多修订.

For a single test, I could probably hack something together with git bisect. But for multiple failed test this is probably not sufficient, since we need to search in both directions for many revisions.

推荐答案

git

您是否正在使用 git 或 Mercurial bisect有什么好处?)?

git

Are you using git or mercurial (see: What is Mercurial bisect good for?)?

假设项目的2.6.18版可以正常工作，但是"master"上的版本崩溃了.有时，找到导致这种回归的原因的最佳方法是对项目的历史记录进行蛮力搜索，以找出引起问题的特定提交. git bisect命令可以帮助您做到这一点:[...]

Suppose version 2.6.18 of your project worked, but the version at "master" crashes. Sometimes the best way to find the cause of such a regression is to perform a brute-force search through the project's history to find the particular commit that caused the problem. The git bisect command can help you do this:[...]

来自:发现问题-Git Bisect .

基本上，您将两个修订标记为开始和结束，然后点击:

Essentially you mark two revisions as start and end and hit:

$ git bisect

然后运行测试，并取决于是通过还是失败了呼叫

Then run the test and depending on whether it passed or failled call

$ git bisect good

或

$ git bisect bad

分别. git会进行二进制搜索，总是将剩余的修订版本减半.我想您可以轻松编写脚本.如果您使用的是 svn 的问题，则git可以轻松导入整个存储库.

Respectively. git will do binary search, always cutting remaining revisions in half. I guess you can script it easily. If you are using svn, git can easily import the whole repository.

这不是一个答案，而是一个建议-只需测试每个提交即可！今天，您可以轻松地对连续集成服务器进行群集，使用服务器场可以轻松测试所有提交.

This is not an answer but an advice - just test every single commit! Today you can cluster continuous integration servers easily, with a farm of servers it is not that hard to test all commits.

这篇关于查找未通过测试的提交，而无需在每次提交上都运行每个测试的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！