Jenkins中的随机测试失败 [英] Random Test Failures in Jenkins

问题描述

我在Python-Django网站上工作.我使用GIT作为我的VCS.为了进行持续集成，我使用Jenkins CI.我已经使用Python设置了两个虚拟环境，一个用于开发，另一个用于预生产.

I work on Python - Django websites. I use GIT as my VCS. For continuous Integration, I use Jenkins CI. I have set up two virtual environments using Python, one for development and other for pre-production.

我的问题: 我为网站编写了许多单元测试，回归测试和冒烟测试.我的开发和生产前虚拟环境都连接到Jenkins CI.

My issue: I have many unit tests, regression and smoke tests written for the website. both my development and pre-production virtualenvs are connected to the Jenkins CI.

最近，每当将代码更改推送到Jenkins CI中的两个环境时，测试都会随机失败.有时，测试会随机失败，而无需进行任何代码更改.

Recently, tests are failing randomly for both the environments in Jenkins CI whenever the changes in code are pushed to them. Sometimes, tests are failing randomly without any code changes been pushed forward.

已完成故障排除:

• 在本地运行测试，它们通过了.
• 通过测试的Jenkins CI(使用立即构建"按钮)是否手动构建了一些构建.
• 分别测试失败的测试，但仍可以通过.

在较早版本中失败的测试在下一个版本中通过.在较早的版本中通过的某些测试在接下来的版本中失败了.有人可以建议我可以做什么吗?

The tests that failed in the earlier builds passed in the next builds. And some tests that passed in the earlier builds, failed in the next builds. Can someone suggest what I can do?

推荐答案

您将不得不确定导致测试随机失败的环境因素.

You are going to have to identify an environmental factor that causes the tests to fail randomly.

我所见的某些原因是这样的:

Some things I have seen cause this:

• 内存-CI机器上还有其他东西正在运行 没有足够的内存来完成所有任务并构建您的东西
• 时间-您的代码中某些内容可能会失败，具体取决于 时间.例如，我的代码将在2月29日失败.它 在五月成功之后使我们感到惊讶.可能是像 如果只有一位数字，则无法格式化秒数.
• 外部依赖关系-您的测试依赖于其他一些服务器 向上.如果它掉下来或变得非常忙，它不会响应您的 测试代码，测试失败.这可能是数据库服务器.
• 数据库内容-您可能没有为针对数据库运行的测试正确设置所有前提条件
• 并发性-有时，多线程代码仅在条件正确(或错误)的情况下才会失败.一种 由外部因素引入的很少的随机延迟可以使代码 工作或使其失败.它很容易忽略比赛条件 多线程代码.
• 服务器(或CPU)-有时测试会失败 如果它运行在测试计算机中的特定服务器或核心上. 当然，如果只有一台测试机，则不会发生这种情况.但 如果一台机器发生故障，则连接不良(防火墙 规则)，正在运行的其他进程，更少(或更多)的内存，您的测试 随机分配它们在该计算机上运行时，可能会失败.
• [在这里插入您的内容]-还有一百万.

• Memory - there are other things running on the CI machine and it doesn't have enough memory to do all of them and build your stuff
• Time - There is something in your code that fails depending on the time. For example, I had code that would fail on Feb 29th. It surprised us after succeeding may times. It could be something like a failure to format the number of seconds if there was only one digit.
• External dependencies - Your tests depend on some other server to be up. If it goes down or gets really busy, it won't respond to your test code and the test fails. This could be a database server.
• Database content - You might not have set all the preconditions correctly for the test that runs against the database
• Concurrency - Sometimes multi-threaded code will only fail when conditions are just right (or just wrong). A little random delay introduced by outside factors could make the code work or make it fail. Its easy to overlook race conditions in multi-threaded code.
• Servers (or CPUs) - Sometimes a test will fail if it runs on a particular server or core in among the test machines. Of course if you only have one test machine, this can't happen. But if one machine has something broken, poor connectivity (firewall rules), other processes running, less (or more) memory, your tests could fail when they are randomly assigned to run on that one.
• [Insert yours here] - And there are a million more.

这些是很难解决的问题.特别是如果他们无缘无故地离开.它使您感到紧张，因为您怀疑当您急着要修复生产系统中的一个讨厌的错误时，它会再次出现.

These are hard problems to solve. Especially if they go away for no good reason. It makes you nervous because you suspect it will come back just when you are in a big hurry to fix a nasty bug in the production system.

这篇关于Jenkins中的随机测试失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！