如何调试随机黄瓜失败？ [英] How to debug random cucumber failures?
问题描述
我遇到了黄瓜测试套件的问题，我无法想到任何调试方式。
I'm stuck with an issue with a cucumber test suite and I can't think of any way of debugging it.
我们有一个很大的套装黄瓜功能，所有这些都通过开发机器。问题是，当我们在我们的ci服务器上运行整个黄瓜套件，并且运行它们单独使它们通过和失败（显然）随机当情况试图填充一个形式（显然不是页）。因为随机失败我认为这是一个时间问题与ajax请求，但似乎不是这样，因为添加真正的大睡眠（尝试从1到60秒的一切）不会改变任何东西。这种情况更有趣，因为有另外3个场景运行相同的步骤，在同一个顺序中第一个失败，这些通过除非我删除第一个场景，在这种情况下，运行这些步骤的第一个是一个失败。
We have a pretty sizeable suit of cucumber features, all of them pass on the development machines. The problem is a couple of scenarios are failing when we run the whole cucumber suite on our ci server, and running them individually makes them pass and fail (apparently) randomly when the scenario tries to fill a form (that apparently isn't on the page). Because of the random failures I thought it was a timing problem with an ajax request, but that doesn't seem to be the case, because adding really big sleep (tried everything from 1 to 60 secs) doesn't change anything. This scenario is even more fun because there's another 3 scenearios running the same steps that are failing on the first one in the same order, and these pass except if I delete the first scenario, in which case the first one to run those steps is the one that fails.
有没有什么技巧调试这种奇怪的黄瓜？功能（记住这些场景总是传递给dev机器，问题是在ci服务器中）。
Is there any tricks to debug this sort of weirdness on cucumber? features (keep in mind these scenarios always pass on the dev machines, the problem is in the ci server).
谢谢！
推荐答案
我也有机会调试间歇性测试失败，只能在CI上重现。在我的经验中，问题总是归结为几个基本原因：
I've also had the opportunity to debug intermittent test failures that only reproduce on CI. In my experience the problem always boiled down to few basic causes:
- 前端的比赛条件。例如，在xhr回调之前启用输入表单会添加默认值。
- 乐观从前端写入。有时，前端工程师通过忽略结果使涉及PUT / POST请求的操作更灵敏。在这种情况下，没有办法让Cucumber等待，直到请求完成，所以对数据库中的状态更改的测试将与应用程序竞争。
- 请求资源在测试夹具中不可用。例如，对第三方API的请求可能会被CI阻止。有时，URL在测试环境中不能正确构造，特别是在手动构建时，而不是使用Rails帮助程序。
间歇性黄瓜故障总是具有挑战性调试。不要放弃！值得努力去弄清楚如何构建一个可测试的，无竞争的前端。您可以使用capybara-webkit来调试仅CI故障。获取javascript控制台输出打印出的CI，然后你可以添加打印到您的javascript以跟踪其状态直到测试失败的点。您还可以破解capybara-webkit打印出前端请求的信息。以下是一个示例： https://github.com/joshuanapoli/capybara-webkit/commit/ 96b645073f7b099196c5d3da4653606e98a453e4
Intermittent Cucumber failures are always challenging to debug. Don't give up! It's worth the effort to figure out how to build a testable, race-free front-end. You can use capybara-webkit to debug the CI-only failures. Get the javascript console output to printed out on CI, and then you can add prints to your javascript to trace its state up to the point of test failure. You can also hack capybara-webkit to print out information about requests made by the front-end. Here is an example: https://github.com/joshuanapoli/capybara-webkit/commit/96b645073f7b099196c5d3da4653606e98a453e4
这篇关于如何调试随机黄瓜失败？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
