iOS 测试/规范 TDD/BDD 和集成 &验收测试 [英] iOS Tests/Specs TDD/BDD and Integration & Acceptance Testing

问题描述

在 iPhone 上进行行为驱动开发的最佳技术是什么?有哪些开源示例项目展示了这些技术的合理使用?以下是我发现的一些选项:

What are the best technologies to use for behavior-driven development on the iPhone? And what are some open source example projects that demonstrate sound use of these technologies? Here are some options I've found:

1. OCUnit/SenTestingKit 如iOS 开发指南:单元测试应用程序 &其他 OCUnit 参考.
   • 示例:iPhoneUnitTests, Three20

1. OCUnit/SenTestingKit as explained in iOS Development Guide: Unit Testing Applications & other OCUnit references.
   • Examples: iPhoneUnitTests, Three20

RSpec 风格

1. Kiwi(也带有嘲讽和期望)
2. 雪松
3. Jasmine 和 UI 自动化 如灵巧的 iOS 验收测试规范

1. Kiwi (which also comes with mocking & expectations)
2. Cedar
3. Jasmine with UI Automation as shown in dexterous' iOS-Acceptance-Testing specs

<小时>

验收测试

Selenium 风格

1. UI 自动化(适用于设备)

1. UI Automation (works on device)

• UI 自动化仪器指南
• UI 自动化参考文档
• Tuneup js - 与 UIAutomation 一起使用的酷库.

• 将用户界面操作捕获到自动化脚本中

• UI Automation Instruments Guide
• UI Automation reference documentation
• Tuneup js - cool library for using with UIAutomation.

• Capturing User Interface Actions into Automation Scripts

可以使用 Cucumber(写在 JavaScript 中) 来驱动 UI 自动化.这将是一个伟大的开源项目.然后，我们可以编写 Gherkin 来运行 UI 自动化测试.现在，我只会将 Gherkin 写为评论.

It's possible to use Cucumber (written in JavaScript) to drive UI Automation. This would be a great open-source project. Then, we could write Gherkin to run UI Automation testing. For now, I'll just write Gherkin as comments.

更新:Zucchini Framework 似乎融合了 Cucumber &用户界面自动化！:)

UPDATE: Zucchini Framework seems to blend Cucumber & UI Automation! :)

旧博客帖子:

• Alex Vollmer 的 UI 自动化教程
• O'Reilly Answers UI 自动化教程
• Adi Saxena 的 UI 自动化教程

UISpec 和 UISpecRunner

• UISpec 是 Google 代码上的开源.
• UISpec 有全面的文档.

FoneMonkey

黄瓜风格

1. Frank 和 iCuke(基于 Cucumber遇到 iPhone 谈话)

• Frank Google Group 的活动比 iCuke Google 群组.
• Frank 同时在设备和模拟器上运行，而 iCuke 只在模拟器中运行.
• Frank 似乎有一套比 iCuke 的步骤定义.而且，Frank 在他们的 wiki 上还有一个步骤定义纲要.
• 我没有提议我们合并 iCuke &Frank(类似于 Merb 和 Rails 的合并方式)，因为它们有相同的共同目标:iOS 版 Cucumber.

• The Frank Google Group has much more activity than the iCuke Google Group.
• Frank runs on both device and simulator, while iCuke only runs in simulator.
• Frank seems to have a more comprehensive set of step definitions than iCuke's step definitions. And, Frank also has a step definition compendium on their wiki.
• I proposed that we merge iCuke & Frank (similar to how Merb & Rails merged) since they have the same common goal: Cucumber for iOS.

KIF(保持功能) 作者 广场

Zucchini Framework 使用 Cucumber 语法编写测试，使用 CoffeeScript 定义步骤.p>

Zucchini Framework uses Cucumber syntax for writing tests and uses CoffeeScript for step definitions.

补充

• OCMock 用于模拟
• OCHamcrest 和/或 Expecta 期待

Additions

• OCMock for mocking
• OCHamcrest and/or Expecta for expectations

嗯，显然，这个问题没有正确答案，但这是我目前选择的:

Well, obviously, there's no right answer to this question, but here's what I'm choosing to go with currently:

对于单元测试，我曾经在 XCode 4 中使用 OCUnit/SenTestingKit.它是简单&坚硬的.但是，我更喜欢 BDD 的语言而不是 TDD(为什么是 RSpec比 Test::Unit 更好?)因为我们的语言创造了我们的世界.所以现在，我使用 Kiwi with ARC &Kiwi 代码完成/自动完成.我更喜欢 Kiwi 而不是 Cedar，因为它建立在 OCUnit 之上，并带有 RSpec 风格的匹配器和模拟/存根.更新:我现在正在研究 OCMock 因为，目前，Kiwi 不支持存根免费桥接对象.

For unit testing, I used to use OCUnit/SenTestingKit in XCode 4. It's simple & solid. But, I prefer the language of BDD over TDD (Why is RSpec better than Test::Unit?) because our words create our world. So now, I use Kiwi with ARC & Kiwi code completion/autocompletion. I prefer Kiwi over Cedar because it's built on top of OCUnit and comes with RSpec-style matchers & mocks/stubs. UPDATE: I'm now looking into OCMock because, currently, Kiwi doesn't support stubbing toll-free bridged objects.

对于验收测试，我使用 UI 自动化，因为它很棒.它可以让您记录每个测试用例，从而自动编写测试.此外，Apple 开发了它，因此它有一个充满希望的未来.它也适用于设备和 Instruments，它允许其他很酷的功能，比如显示内存泄漏.不幸的是，使用 UI 自动化，我不知道如何运行 Objective-C 代码，但是使用 Frank &iCuke 你可以.因此，我将使用单元测试来测试较低级别的 Objective-C 内容，或者仅为 TEST 构建配置，点击后将运行 Objective-C 代码.

For acceptance testing, I use UI Automation because it's awesome. It lets you record each test case, making writing tests automatic. Also, Apple develops it, and so it has a promising future. It also works on the device and from Instruments, which allows for other cool features, like showing memory leaks. Unfortunately, with UI Automation, I don't know how to run Objective-C code, but with Frank & iCuke you can. So, I'll just test the lower-level Objective-C stuff with unit tests, or create UIButtons only for the TEST build configuration, which when clicked, will run Objective-C code.

您使用哪些解决方案?

• 是否有目前适用于 iOS4 和 Xcode4 的 BDD 解决方案?
• SenTestingKit(与 XCode 集成)) 与 XCode 4 上的 GHUnit 进行单元测试?
• 使用 OCunit 在 iOS 上测试异步代码
• Xcode 4 中的 SenTestingKit:异步测试?
• iPhone 上的单元测试如何工作?

推荐答案

tl;dr

在 Pivotal，我们编写了 Cedar，因为我们在 Ruby 项目中使用并喜欢 Rspec.Cedar 并不是要取代 OCUnit 或与 OCUnit 竞争.它旨在为 Objective C 带来 BDD 风格测试的可能性，就像 Rspec 在 Ruby 中开创 BDD 风格测试一样，但并没有消除 Test::Unit.选择其中一个很大程度上取决于风格偏好.

tl;dr

At Pivotal we wrote Cedar because we use and love Rspec on our Ruby projects. Cedar isn't meant to replace or compete with OCUnit; it's meant to bring the possibility of BDD-style testing to Objective C, just as Rspec pioneered BDD-style testing in Ruby, but hasn't eliminated Test::Unit. Choosing one or the other is largely a matter of style preferences.

在某些情况下，我们设计 Cedar 是为了克服 OCUnit 为我们工作的方式中的一些缺点.具体来说，我们希望能够在测试中使用调试器，从命令行和 CI 构建中运行测试，并获得有用的测试结果文本输出.这些东西可能或多或少对您有用.

In some cases we designed Cedar to overcome some shortcomings in the way OCUnit works for us. Specifically, we wanted to be able to use the debugger in tests, to run tests from the command line and in CI builds, and get useful text output of test results. These things may be more or less useful to you.

在两个测试框架(例如 Cedar 和 OCUnit)之间做出决定归结为两件事:首选风格和易用性.我将从风格开始，因为这只是意见和偏好的问题；易用性往往是一组权衡.

Deciding between two testing frameworks like Cedar and OCUnit (for example) comes down to two things: preferred style, and ease of use. I'll start with the style, because that's simply a matter of opinion and preference; ease of use tends to be a set of tradeoffs.

风格考虑超越了您使用的技术或语言.xUnit 风格的单元测试比 BDD 风格的测试存在的时间要长得多，但后者迅速流行起来，主要是由于 Rspec.

Style considerations transcend what technology or language you use. xUnit-style unit testing has been around for far longer than BDD-style testing, but the latter has rapidly gained in popularity, largely due to Rspec.

xUnit 样式测试的主要优点是它的简单性和广泛采用(在编写单元测试的开发人员中)；几乎任何您可以考虑用其编写代码的语言都有可用的 xUnit 风格的框架.

The primary advantage of xUnit-style testing is its simplicity, and wide adoption (amongst developers who write unit tests); nearly any language you could consider writing code in has an xUnit-style framework available.

与 xUnit 风格相比，BDD 风格的框架往往有两个主要区别:如何构建测试(或规范)，以及编写断言的语法.对我来说，结构差异是主要的区别.xUnit 测试是一维的，对给定测试类中的所有测试都有一个 setUp 方法.然而，我们测试的类不是一维的；我们经常需要在几个不同的、可能相互冲突的环境中测试操作.例如，考虑一个简单的 ShoppingCart 类，它有一个 addItem: 方法(为了这个答案，我将使用 Objective C 语法).当购物车为空时，与购物车包含其他物品时相比，此方法的行为可能有所不同；如果用户输入了折扣代码，则可能会有所不同；如果指定的物品无法通过所选的运输方式运输，则可能会有所不同；等等.当这些可能的条件相互交叉时，你最终会得到数量呈几何级数增加的可能上下文；在 xUnit 样式的测试中，这通常会导致很多方法的名称类似于 testAddItemWhenCartIsEmptyAndNoDiscountCodeAndShippingMethodApplies.BDD 风格框架的结构允许您单独组织这些条件，我发现这样可以更轻松地确保涵盖所有情况，也更易于查找、更改或添加单个条件.例如，使用 Cedar 语法，上面的方法看起来像这样:

BDD-style frameworks tend to have two main differences when compared to xUnit-style: how you structure the test (or specs), and the syntax for writing your assertions. For me, the structural difference is the main differentiator. xUnit tests are one-dimensional, with one setUp method for all tests in a given test class. The classes that we test, however, aren't one-dimensional; we often need to test actions in several different, potentially conflicting, contexts. For example, consider a simple ShoppingCart class, with an addItem: method (for the purposes of this answer I'll use Objective C syntax). The behavior of this method may differ when the cart is empty compared to when the cart contains other items; it may differ if the user has entered a discount code; it may differ if the specified item can't be shipped by the selected shipping method; etc. As these possible conditions intersect with one another you end up with a geometrically increasing number of possible contexts; in xUnit-style testing this often leads to a lot of methods with names like testAddItemWhenCartIsEmptyAndNoDiscountCodeAndShippingMethodApplies. The structure of BDD-style frameworks allows you to organize these conditions individually, which I find makes it easier to make sure I cover all cases, as well as easier to find, change, or add individual conditions. As an example, using Cedar syntax, the method above would look like this:

describe(@"ShoppingCart", ^{
    describe(@"addItem:", ^{
        describe(@"when the cart is empty", ^{
            describe(@"with no discount code", ^{
                describe(@"when the shipping method applies to the item", ^{
                    it(@"should add the item to the cart", ^{
                        ...
                    });

                    it(@"should add the full price of the item to the overall price", ^{
                        ...
                    });
                });

                describe(@"when the shipping method does not apply to the item", ^{
                    ...
                });
            });

            describe(@"with a discount code", ^{
                ...
            });
        });

        describe(@"when the cart contains other items, ^{
            ...
        });
    });
});

在某些情况下，您会在其中找到包含相同断言集的上下文，您可以使用共享的示例上下文来干掉这些断言.

In some cases you'll find contexts in that contain the same sets of assertions, which you can DRY up using shared example contexts.

BDD 风格的框架和 xUnit 风格的框架之间的第二个主要区别，断言(或匹配器")语法，只是使规范的风格更好一些；有些人真的很喜欢它，有些人则不喜欢.

The second main difference between BDD-style frameworks and xUnit-style frameworks, assertion (or "matcher") syntax, simply makes the style of the specs somewhat nicer; some people really like it, others don't.

这导致了易用性的问题.在这种情况下，每个框架都有其优点和缺点:

That leads to the question of ease of use. In this case, each framework has its pros and cons:

• OCUnit 的存在时间比 Cedar 长得多，并且直接集成到 Xcode 中.这意味着创建新的测试目标很简单，而且在大多数情况下，启动和运行测试正常工作".另一方面，我们发现在某些情况下，例如在 iOS 设备上运行，让 OCUnit 测试工作几乎是不可能的.设置 Cedar 规范比 OCUnit 测试需要更多的工作，因为您已经获得了库并自己链接到它(在 Xcode 中绝不是一项微不足道的任务).我们正在努力简化设置，我们非常欢迎您提出任何建议.

• OCUnit has been around much longer than Cedar, and is integrated directly into Xcode. This means it's simple to make a new test target, and, most of the time, getting tests up and running "just works." On the other hand, we found that in some cases, such as running on an iOS device, getting OCUnit tests to work was nigh impossible. Setting up Cedar specs takes some more work than OCUnit tests, since you have get the library and link against it yourself (never a trivial task in Xcode). We're working on making setup easier, and any suggestions are more than welcome.

OCUnit 运行测试作为构建的一部分.这意味着您不需要运行可执行文件来运行您的测试；如果任何测试失败，您的构建就会失败.这使得运行测试的过程更简单，并且测试输出直接进入您的构建输出窗口，使其易于查看.出于以下几个原因，我们选择将 Cedar 规范构建到您单独运行的可执行文件中:

OCUnit runs tests as part of the build. This means you don't need to run an executable to make your tests run; if any tests fail, your build fails. This makes the process of running tests one step simpler, and test output goes directly into your build output window which makes it easy to see. We chose to have Cedar specs build into an executable which you run separately for a few reasons:

• 我们希望能够使用调试器.您可以像运行任何其他可执行文件一样运行 Cedar 规范，因此您可以以相同的方式使用调试器.
• 我们想要简单的控制台登录测试.您可以在 OCUnit 测试中使用 NSLog()，但输出会进入构建窗口，您必须在其中展开构建步骤才能读取它.
• 我们想要易于阅读的测试报告，无论是在命令行还是在 Xcode 中.OCUnit 结果很好地显示在 Xcode 的构建窗口中，但是从命令行构建(或作为 CI 过程的一部分)会导致测试输出与大量其他构建输出混合在一起.通过单独的构建和运行阶段，Cedar 将输出分开，因此测试输出很容易找到.默认的 Cedar 测试运行器复制标准的打印样式.".对于每个通过的规范，F"表示失败的规范，等等.Cedar 还能够使用自定义报告器对象，因此您可以轻松地以任何您喜欢的方式输出结果.

OCUnit 是 Objective C 的官方单元测试框架，得到 Apple 的支持.苹果基本上拥有无限的资源，所以如果他们想完成某件事，它就会完成.毕竟，这是我们正在玩的 Apple 沙箱.然而，硬币的另一面是 Apple 每天收到大量支持请求和错误报告.他们非常擅长处理所有问题，但他们可能无法立即处理您报告的问题，或者根本无法处理.与 OCUnit 相比，Cedar 更新且不太成熟，但如果您有任何疑问、问题或建议，请发送消息至 Cedar 邮件列表 ([e-mail])，我们将竭尽所能为您提供帮助.另外，请随意从 Github (github.com/pivotal/cedar) fork 代码并添加您认为缺少的任何内容.我们将测试框架开源是有原因的.

OCUnit is the official unit testing framework for Objective C, and is supported by Apple. Apple has basically limitless resources, so if they want something done it will get done. And, after all, this is Apple's sandbox we're playing in. The flip side of that coin, however, is that Apple receives on the order of a bajillion support requests and bug reports each day. They're remarkably good about handling them all, but they may not be able to handle issues you report immediately, or at all. Cedar is much newer and less baked than OCUnit, but if you have questions or problems or suggestions send a message to the Cedar mailing list ([e-mail]) and we'll do what we can to help you out. Also, feel free to fork the code from Github (github.com/pivotal/cedar) and add whatever you think is missing. We make our testing frameworks open source for a reason.

在 iOS 设备上运行 OCUnit 测试可能很困难.老实说，我已经有一段时间没有尝试过这个了，所以它可能变得更容易了，但是上次我尝试时我根本无法让任何 UIKit 功能的 OCUnit 测试工作.当我们编写 Cedar 时，我们确保我们可以在模拟器和设备上测试依赖于 UIKit 的代码.

Running OCUnit tests on iOS devices can be difficult. Honestly, I haven't tried this for quite some time, so it may have gotten easier, but the last time I tried I simply couldn't get OCUnit tests for any UIKit functionality to work. When we wrote Cedar we made sure that we could test UIKit-dependent code both on the simulator and on devices.

最后，我们为单元测试编写了 Cedar，这意味着它无法与 UISpec 等项目相提并论.自从我尝试使用 UISpec 已经有一段时间了，但我理解它主要专注于以编程方式驱动 iOS 设备上的 UI.我们特别决定不尝试让 Cedar 支持这些类型的规范，因为 Apple(当时)即将发布 UIAutomation.

Finally, we wrote Cedar for unit testing, which means it's not really comparable with projects like UISpec. It's been quite a while since I tried using UISpec, but I understood it to be focused primarily on programmatically driving the UI on an iOS device. We specifically decided not to try to have Cedar support these types of specs, since Apple was (at the time) about to announce UIAutomation.

这篇关于iOS 测试/规范 TDD/BDD 和集成 &amp;验收测试的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！