测试NG&硒:将测试分为“组"，在每个组中有序运行 [英] TestNG & Selenium: Separate tests into "groups", run ordered inside each group

问题描述

我们使用TestNG和Selenium WebDriver来测试我们的Web应用程序.

We use TestNG and Selenium WebDriver to test our web application.

现在的问题是，我们经常需要按特定顺序运行多个测试，例如:

Now our problem is that we often have several tests that need to run in a certain order, e.g.:

• 登录到应用程序
• 输入一些数据
• 编辑数据
• 检查是否正确显示

现在显然，这些测试需要以精确的顺​​序运行.

Now obviously these tests need to run in that precise order.

与此同时，我们还有许多其他测试，这些测试完全独立于上面的测试列表.

At the same time, we have many other tests which are totally independent from the list of tests above.

因此，我们希望能够以某种方式将测试放入组"(不一定是TestNG意义上的组)，然后以如下方式运行它们:

So we'd like to be able to somehow put tests into "groups" (not necessarily groups in the TestNG sense), and then run them such that:

• 一个组"内的测试始终以相同的顺序运行
• 但不同的测试组"作为一个整体可以按任何顺序运行

第二点很重要，因为我们要避免不同组中测试之间的依赖性(因此可以独立使用和开发不同的测试组".)

The second point is important, because we want to avoid dependencies between tests in different groups (so different test "groups" can be used and developed independently).

是否可以使用TestNG实现此目的?

Is there a way to achieve this using TestNG?

我们尝试过的解决方案

• 首先，我们只是将属于同一类的测试放在一个类中，并使用dependsOnMethods使其以正确的顺序运行.这曾经在TestNG V5中起作用，但是在V6中，TestNG有时会交错来自不同类的测试(同时遵守dependsOnMethods施加的顺序).似乎没有一种方法可以告诉TestNG始终一起运行一个类的测试".
• 我们考虑编写方法拦截器.但是，这样做的缺点是从IDE内部运行测试变得更加困难(因为直接在类上调用测试不会使用拦截器).另外，拦截器无法对使用dependsOnMethods的测试进行排序，因此我们必须停止使用它.我们可能必须创建自己的注释来指定顺序，并且我们希望尽可能使用标准的TestNG功能.
• TestNG文档建议使用preserve-order来订购测试.这看起来很有希望，但是只有在您分别列出每种测试方法时才可行，这似乎是多余的且难以维护.

• At first we just put tests that belong together into one class, and used dependsOnMethods to make them run in the right order. This used to work in TestNG V5, but in V6 TestNG will sometimes interleave tests from different classes (while respecting the ordering imposed by dependsOnMethods). There does not seem to be a way to tell TestNG "Always run tests from one class together".
• We considered writing a method interceptor. However, this has the disadvantage that running tests from inside an IDE becomes more difficult (because directly invoking a test on a class would not use the interceptor). Also, tests using dependsOnMethods cannot be ordered by the interceptor, so we'd have to stop using that. We'd probably have to create our own annotation to specify ordering, and we'd like to use standard TestNG features as far as possible.
• The TestNG docs propose using preserve-order to order tests. That looks promising, but only works if you list every test method separately, which seems redundant and hard to maintain.

有没有更好的方法来实现这一目标?

Is there a better way to achieve this?

我也愿意接受关于如何处理彼此构建的测试的任何其他建议，而不必对 all 个测试强加全部命令.

I am also open for any other suggestions on how to handle tests that build on each other, without having to impose a total order on all tests.

PS

alanning的答案指出，通过在每个测试中进行必要的设置，我们可以简单地使所有测试保持独立.从原则上讲，这是一个好主意(有些测试可以做到这一点)，但是有时候我们需要测试一个完整的工作流程，每个步骤都取决于之前的所有步骤(如我的示例).要使用独立"测试来做到这一点，将意味着一遍又一遍地运行相同的多步骤设置，这将使我们本来就很慢的测试变得更加慢.而不是执行三个测试:

alanning's answer points out that we could simply keep all tests independent by doing the necessary setup inside each test. That is in principle a good idea (and some tests do this), however sometimes we need to test a complete workflow, with each step depending on all previous steps (as in my example). To do that with "independent" tests would mean running the same multi-step setup over and over, and that would make our already slow tests even slower. Instead of three tests doing:

• 测试1:登录到应用程序
• 测试2:输入一些数据
• 测试3:编辑数据

我们会得到

• 测试1:登录到应用程序
• 测试2:登录到应用程序，输入一些数据
• 测试3:登录到应用程序，输入一些数据，然后编辑数据 等

• Test 1: login to application
• Test 2: login to application, enter some data
• Test 3: login to application, enter some data, edit the data etc.

除了不必要地增加测试时间外，这还感觉不自然-应该可以将工作流建模为一系列测试.

In addition to needlessly increasing testing time, this also feels unnatural - it should be possible to model a workflow as a series of tests.

如果没有其他方法，这可能就是我们的处理方式，但是我们正在寻找一种更好的解决方案，而无需重复相同的设置调用.

If there's no other way, this is probably how we'll do it, but we are looking for a better solution, without repeating the same setup calls.

推荐答案

您正在混合功能"和测试".分离它们将解决您的问题.

You are mixing "functionality" and "test". Separating them will solve your problem.

例如，创建一个执行登录步骤的助手类/方法，然后在您的Login测试和要求用户登录的所有其他测试中调用该类/方法.

For example, create a helper class/method that executes the steps to log in, then call that class/method in your Login test and all other tests that require the user to be logged in.

您的其他测试实际上并不需要依赖您的登录测试"，只需登录类/方法即可.

Your other tests do not actually need to rely on your Login "Test", just the login class/method.

如果以后的后端修改在登录过程中引入了错误，则所有依赖于登录帮助程序类/方法的测试仍将按预期失败.

If later back-end modifications introduce a bug in the login process, all of the tests which rely on the Login helper class/method will still fail as expected.

更新:

结果证明这已经有一个名称，即页面对象"模式.这是包含使用此模式的Java示例的页面:

Turns out this already has a name, the Page Object pattern. Here is a page with Java examples of using this pattern:

http://code.google.com/p/selenium/wiki/PageObjects

这篇关于测试NG&硒:将测试分为“组"，在每个组中有序运行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！