Selenium + JUnit：测试订单/流量？ [英] Selenium + JUnit: test order/flow?
问题描述
我正在使用Selenium来测试我的java web应用程序的html页面（实际上是JSP）。我的网络应用程序需要一个流程来访问每个页面（它是一个小型的在线游戏网络应用程序），如：要进入页面B，您需要转到页面A，输入一些文本并按一个按钮进入页面B.显然我已经有一些测试来验证页面A是否正常工作。
I am using Selenium to test my java web app's html pages (JSPs actually). My web app requires a flow to access each pages (it is a small online game web app), as in: to get to page B you'd need to go to page A, enter some text and press a button to get to page B. Obviously I already have some tests to verify that page A is working properly.
我希望能够编写更多测试以便在测试后检查对于页面A运行，我将获得运行页面B的测试（对于应用程序的其余部分依此类推）。所以简而言之：在我的测试中以某种方式定义一些顺序。
I would like to be able to write more tests in order to check that after the tests for page A run I'll get my tests for page B running (and so on for the rest of the app). So in short: define some order in my tests somehow.
在过去几天做了很多关于测试的阅读之后，我找不到关于这个特定主题的任何有趣的东西。因此，我现在就征求意见。
After doing lots of reading about testing in the last few days, I cannot find anything interesting on this specific subject. Hence I'm asking for advice now.
我已经确定的可能的解决方案：
Possible Solutions I have identified:
-
为页面A定义（在同一测试类中）测试方法，然后测试测试B的测试方法。然后命令执行测试方法。但我们知道JUnit（但TestNG确实）不允许测试方法执行顺序，请参阅 SO问题selenium-junit-tests-how-do-do-do-do-do-tests-in-a-test-in-sequential-order
在一种测试方法下对所有测试（对于第A页，第B页，等等）进行分组。但我读到它很糟糕，请参阅所以问题：junit-one-test-case-per-method-or-multiple-test-cases-per-methods 。做硒测试时是不是很糟糕？我已经看到一些代码这样做，所以我认为它可能不是。
Grouping all the tests (for page A, page B, & so on) under one test method. But I've read it's bad, see SO question: junit-one-test-case-per-method-or-multiple-test-cases-per-method. Is it that bad when doing selenium test? I've seen some code doing it so I assume it may not be.
对所有测试进行分组（对于第A页，第B页，等等） ）在一个测试方法下，但使用JUnit的ErrorCollector类： ErrorCollector 允许您以相同的方法执行有序检查，并在出现故障时产生特定的错误消息但是让方法（因此检查）一直运行到结束。这个解决方案对我来说似乎太野蛮。
Grouping all the tests (for page A, page B, & so on) under one test method but use the JUnit's ErrorCollector class: ErrorCollector allows you to execute ordered checks in the same method and yields a specific error message if one fails BUT let the method (hence the checks) running until the end. This solution seems too "brutal" to me.
使用JUnit的TestSuite类：它运行测试套件中列出的测试，测试类在套件中定义。因此，这将涉及在测试类中测试页面A的独立测试方法（比如说TestA），然后测试测试类中的页面B的所有测试方法（比如说TestB），依此类推。然后将它们插入测试套件中，例如@SuiteClasses（{TestA.class，TestB.class，TestC.class，...}）
Use JUnit's TestSuite class: it runs test listed in the suite in the order test classes are defined in the suite. So that would involve having independent test methods to test page A in a test class (let's say TestA), then all the test methods to test page B in a test class (let's say TestB), and so on. Then insert those in a test suite such as @SuiteClasses( {TestA.class, TestB.class, TestC.class, ...} )
使用< a href =http://www.junit.org/apidocs/junit/framework/TestSuite.html =nofollow noreferrer> JUnit的TestSuite类与JUnit的ErrorCollector类相结合。哦，好吧，既然我们可以，你可能想要在不同的课程中每页进行分组测试，&在该组页面顶部使用ErrorCollector测试区域。如果您有非常密集的网页或其他原因，此解决方案可能非常有用。
Use JUnit's TestSuite class in combination with JUnit's ErrorCollector class. Oh well, since we can, you may want to Group test per page in different classes, & on top of that group page tests "zones" using ErrorCollector. This solution may be very usefull if you have a very dense web page or other reasons.
相当激进：使用其他工具，例如 TestNG 可以访问测试方法排序等功能。
Quite radical: Use another tool such as TestNG to have access to features such as test method ordering.
注意：我想有些人会推荐最后一个解决方案（迁移到TestNG），但我也希望听到与JUnit相关的其他想法/意见。如果我在一个不能（由于某种原因）迁移到另一个测试框架的团队中工作，那么他们将如何处理这个测试订购问题呢？
Note: I imagine some would recommend the last solution (migrate to TestNG) but I'd like to hear other ideas/opinions tied to JUnit too. As in, if I'm working in a team that is not able (for some reason) to migrate to another testing framework, then how would they address this test ordering issue?
推荐答案
为什么要迁移？您可以使用JUnit进行单元测试，使用另一个框架进行更高级别的测试。在您的情况下，它是一种接受或功能或端到端，如何命名它并不重要。但重要的是要了解这些测试不是单位。他们坚持不同的规则：它们更复杂，运行时间更长，更少，它们需要复杂的设置，外部依赖性，并且可能偶尔会失败。为什么不为它们使用另一个框架（甚至是另一种编程语言）？
Why to migrate? You can use JUnit for unit-testing and another framework for higher-level testing. In your case it is a kind of acceptance or functional or end-to-end, it is not that important how you name it. But important is to understand that these tests are not unit. They stick to different rules: they are more complex, run longer and less often, they require complex setup, external dependencies and may sporadically fail. Why not use another framework for them (or even another programming language)?
可能的变体是：
- BDD frameworks: already mentioned Cucumber, JDave, JBehave, Spock. For instance, Spock is JUnit based and Groovy =)
- TestNG
如果添加另一个框架不是一个选项：你为JUnit列举了更多选项然后我可以想象=）我会把在一个测试方法中流程的整个测试脚本，并将测试代码组织到驱动程序中。这意味着您的端到端测试不会直接调用您的应用程序或Selenium API的方法，而是将它们包装到Driver组件的方法中，这些方法隐藏了API的复杂性，看起来像是发生了什么或预期的语句。看一下这个例子：
If adding another framework is not an option: you enumerated more options for JUnit then I could imagine =) I would put the whole test script for the flow in one test method and would organize test code into "Drivers". That means that your end-to-end tests do not call the methods of your application or Selenium API directly, but wrap them into methods of Driver components which hide API complexity and look like statements of what happens or what is expected. Look at the example:
@Test
public void sniperWinsAnAuctionByBiddingHigher() throws Exception {
auction.startSellingItem();
application.startBiddingIn(auction);
auction.hasReceivedJoinRequestFrom(ApplicationRunner.SNIPER_XMPP_ID);
auction.reportPrice(1000, 98, "other bidder");
application.hasShownSniperIsBidding(auction, 1000, 1098);
auction.hasReceivedBid(1098, ApplicationRunner.SNIPER_XMPP_ID);
auction.reportPrice(1098, 97, ApplicationRunner.SNIPER_XMPP_ID);
application.hasShownSniperIsWinning(auction, 1098);
auction.announceClosed();
application.hasShownSniperHasWonAuction(auction, 1098);
}
摘录自以测试为导向的面向对象软件的发展。这本书真的很棒，我强烈建议阅读它。
A snippet is taken from the "Growing Object-Oriented Software Guided by Tests". The book is really great and I highly recommend to read it.
这是真正的端到端测试，它使用真正的XMPP连接，Openfire jabber服务器和WindowLicker Swing GUI测试框架。但所有这些东西如果卸载到驱动程序组件。在你的测试中，你只看到不同的演员如何沟通。并且订购：在申请开始竞标之后我们检查拍卖服务器收到加入请求，然后我们指示拍卖服务器报告新价格并检查它是否反映在UI等。整个代码可在 github 。
This is real end-to-end test that uses real XMPP connection, Openfire jabber server and WindowLicker Swing GUI-testing framework. But all this stuff if offloaded to Driver components. And in your test you just see how different actors communicate. And it is ordered: after application started bidding we check that auction server received join request, then we instruct auction server to report new price and check that it is reflected in UI and so on. The whole code is available on github.
github上的示例很复杂，因为应用程序并不像书籍示例中常见的那样简单。但是那本书逐渐给出了它，我能够按照书籍指南从头开始构建整个应用程序。事实上，这是我读过的关于TDD和自动化开发人员测试的唯一一本书，它提供了如此全面而完整的示例。而且我读了很多。但请注意，驱动程序方法不会使您的测试单元。它只是让你隐藏复杂性。它也可以（也应该）与其他框架一起使用。如果需要，它们只是为您提供了将测试分成连续步骤的其他可能性;编写用户可读的测试用例;将测试数据外部化为CSV，Excel表格，XML文件或数据库，以使测试超时;与外部系统，servlet和DI容器集成;定义和运行单独的测试组;提供更多用户友好的报告等等。
The example on github is complex, because the application is not as trivial as it usually happens with book examples. But that book gives it gradually and I was able to built the whole application from scratch following the book guide. In fact, it is the sole book I ever read on TDD and automated developer testing that gives such a thorough and complete example. And I've read quite a lot of them. But note, that Driver approach does not make your tests unit. It just allows you hide complexity. And it can (and should) be used with other frameworks too. They just give you additional possibilities to split your tests into sequential steps if you need; to write a user readable test cases; to externalize test data into CSV,Excel tables, XML files or database, to timeout your tests; to integrate with external systems, servlet and DI containers; to define and run separately test groups; to give more user-friendly reports and so on.
关于制作所有测试单元。对于数学，字符串处理等实用程序库之类的东西是不可能的。如果您的应用程序完全经过单元测试，则表示您不是测试所有应用程序，也不知道哪些测试是单元测试，哪些测试不是。第一种情况可能没问题，但是未涵盖的所有内容必须由开发人员，测试人员，用户或任何人手动测试和重新测试。这是很常见的，但最好是有意识的决定而不是随意的决定。为什么你不能单元测试一切？
And about making all your tests unit. It is not possible for anything excluding something like utility libraries for math, string processing and so on. If you have application that is completely unit tested that it means either that you test not all application or you do not understand what tests are unit and what are not. The first case may be OK, but everything that is not covered must be tested and retested manually by developers, testers, users or whoever. It is quite common but it better to be conscious decision rather than casual one. Why you cannot unit test everything?
有很多单元测试的定义，它导致了神圣的战争）我更喜欢以下内容：单元测试是单独测试程序单元。有人说：嘿，单位是我的应用！我测试登录，它是简单的单位功能。但也有隐藏在隔离中的语用学。为什么我们需要将单元测试与其他测试区别开来？因为这是我们的第一个安全网。他们必须快。您经常提交（例如git）并在每次提交之前至少运行 。但想象一下，单位测试需要5分钟才能运行。您可以较少地运行它们，也可以减少运行次数，或者您当时只运行一个测试用例甚至一种测试方法，或者等待每2分钟一次，以便在5分钟内完成测试。在5分钟内你将去Coding Horror，在那里你将花费接下来的2个小时=）并且单元测试必须永远不会失败。如果他们这样做 - 你就不会相信他们。因此，隔离：您必须从单元测试中隔离缓慢和偶发故障的来源。因此，隔离意味着不应使用单元测试：
There are a lot of definitions of unit tests and it leads to holy war) I prefer the following: "Unit test is test for program unit in isolation". Some people say: "hey, unit is my application! I test login and it is simple unit function". But there is also pragmatics that hides in isolation. Why do we need to differ unit tests from others? Because it is our first safety net. They must be fast. You commit often (to git, for example) and you run them at least before each commit. But imagine, that "unit" tests takes 5 minutes to run. You will either run them less often or you will commit less often or you will run just one test case or even one test method at the time, or you will wait say each 2 minutes for tests to complete in 5 minutes. An in that 5 minutes you'll go to Coding Horror where you'll spend the next 2 hours =) And unit tests must never fail sporadically. If they do that - you will not trust them. Hence, the isolation: you must isolate slowness and sources of sporadic failures from your unit tests. Hence, isolation means that unit tests should not use:
- 文件系统
- 网络，套接字， RMI等等
- GUI
- 多线程
- 外部库需要测试框架并支持Hamcrest等简单库
- File system
- Network, Sockets, RMI and so on
- GUI
- Multithreading
- External libraries expecting test framework and supporting simple libraries like Hamcrest
单元测试必须是本地的。当您在编码后2分钟内出现缺陷时，您希望只有一个左右的测试失败，而不是整个套件的一半。这意味着您在单元测试中测试有状态行为非常有限。您不应该进行使5个状态转换达到前提条件的测试设置。因为第一次转换中的失败将至少打破4次以下转换的测试，以及您当前为第6次转换编写的另一次测试。任何非平凡的应用程序都有相当多的流和状态转换。所以这不能进行单元测试。出于同样的原因，单元测试不得在数据库，静态字段，Spring上下文或其他任何内容中使用可更改的共享状态。这正是JUnit为每个测试方法创建测试类的新实例的原因。
And unit tests must be local. You want to have just one or so tests failing when you've made a defect within 2 minutes of coding, not a half of the whole suite. That means that you are very limited in testing stateful behavior in unit tests. You should not make a test setup that makes 5 state transitions to reach preconditions. Because fail in first transition will break at least 4 tests for following transitions and one more test that you currently write for the 6th transition. And any non-trivial application has quite a lot of flows and state transitions in it. So this cannot be unit tested. For the same reason unit tests must not use changeable shared state in database, static fields, Spring context or whatever. This is exactly the reason why JUnit creates new instance of test class for every test method.
所以，你看，无论你如何重新编码，你都无法完全对网络应用进行单元测试。因为它有流，JSP，servlet容器，可能更多。当然，你可以忽略这个定义，但它很有用）如果你同意区分单元测试和其他测试是有用的，并且这个定义有助于实现那个，那么你将寻求另一个框架或至少另一种方法如果不是单元测试，您将为不同类型的测试创建单独的套件，依此类推。
So, you see, you cannot fully unit test a web app, no matter how you recode it. Because it has flows, JSPs, servlet container and probably more. Of course, you can just ignore this definition, but it is damn useful) If you agree that distinguishing unit tests from other tests is useful, and this definition helps to achieve that then you'll go for another framework or at least another approach for tests that are not unit, you'll create separate suites for separate kinds of test and so on.
希望，这会有所帮助）
这篇关于Selenium + JUnit：测试订单/流量？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
