集成测试:模拟外部 API 与使用外部 API 沙箱 [英] Integration testing: Mock external API vs. use external API sandbox

问题描述

我们需要使用外部合作伙伴的 API.API 状态良好，我们可以使用沙盒环境进行自动测试.

We're required to use the API of an external partner. The API is in a good shape and we got access to a sandbox environment we can use for automatic testing.

我们已经使用单元测试测试了外部 API 的每一次调用，但在涉及外部合作伙伴方面的复杂操作时，不确定集成测试的最佳实践.

We already test every single call of the external API using unit tests but are unsure regarding best practices for integration tests when it comes to complex operations on the side of the external partner.

示例:我们服务的每个用户也在我们的外部合作伙伴处获得了一个用户对象.当对这个用户对象执行外部 API 调用 X 时，我们希望对象 Y 出现在这个用户的集合 Z 中(我们必须使用不同的调用来查询).

Example: Every user of our service also got a user object at our external partner. When performing external API call X on this user object, we expect object Y to appear inside collection Z of this user (which we have to query using a different call).

此类测试用例的最佳做法是什么?

What are best practices for testing cases like this?

• 尽可能模拟外部 API 并依靠单元测试来完成它们的工作?优点:测试运行速度快且独立于互联网连接.缺点:我们的模拟中的错误可能会导致误报.

• Mock the external API as much as possible and rely on the unit tests to do their job? Advantages: Tests run fast and independent from an internet connection. Disadvantages: Mistakes is in our mocks could lead to false positives.

集成外部 API 沙箱并针对它运行每个集成测试.优点:接近现实生活中的 API 交互.缺点:测试只能在开放的互联网连接下运行，并且需要更多时间.

Integrate the external API sandbox and run every integration test against it. Advantages: Close to real life API interactions. Disadvantages: Tests can only be run with an open internet connection and take more time.

使用模拟和沙盒数据的混合，设置一个布尔值以在需要时在内部(=模拟)和外部(=沙盒)环境之间切换.优点: 可靠的测试.缺点:设置起来可能很麻烦.

Use a hybrid of mocked and sandbox data, set a boolean to switch between the internal (=mocked) and external (=sandbox) environment when required. Advantages: Reliable tests. Disadvantages: Could be a pain to set up.

其他最佳做法?

谢谢！

相关:如何编写用于交互的集成测试使用外部 API? 然而，答案是你没有.你必须真正相信实际的 API 确实有效."我们认为这还不够.

Related: How are integration tests written for interacting with external API? However, the answer "You don't. You have to actually trust that the actual API actually works." is not sufficient in our opinion.

我们担心仅针对我们假设外部 API 应该如何工作(即使它们基于单元测试)而不针对实际 API 进行集成测试，会给我们留下误报.我们需要的是一个测试来验证我们的假设(模拟)实际上是正确的——不仅在单元测试的上下文中，而且在包含多个步骤的复杂操作的上下文中.

We fear that integration testing only against our assumptions how the external API should work (even if they are based on unit tests) – and not against the actual API – will leave us with false positives. What we'd need is a test that verifies that our assumptions (mocks) are actually correct – not only in the context of unit tests but also in the context of complex operations with several steps.

验证可能是一个很好的例子:如果我们弄乱了集成代码并发送格式错误的数据或在我们发送的上下文中没有任何意义的数据，因为我们错过了一个步骤怎么办?我们的模拟 API 不验证(或仅在非常有限的范围内)仍会返回有效数据，而不是传递我们从真实 API 收到的错误.

Validation might be a good example: What if we mess up the integration code and send malformed data or data that does not make any sense in the context we send it in because we missed a step? Our mock API, which does not validate (or only in very limited range) would still return valid data instead of passing the error we would receive from the real API.

推荐答案

我认为当我们与外部 API 交互时，我们需要进行 2 级验证:

I believe there should be 2 level of verifications we need to do when we interface with an external API:

• API 验证:验证 API 是否符合其规范和/或我们的理解
• 应用功能验证:验证我们的业务逻辑是否符合对通过 API 验证的 API 的期望

在我们的例子中，我们使用模拟 API 以及真实和模拟 API 验证.

In our case, we use a mock API together with real and mock API verification.

• Mock API 允许我们仅隔离应用功能的任何运行时错误/异常，因此我们不会将问题归咎于任何外部方
• 针对真实 API 和模拟 API 执行相同的 API 验证，以确保真实 API 以我们期望的方式工作，以及模拟 API 应正确模仿真实 API

如果在此过程中，外部 API 发生变化，API 验证可能会变红，从而触发模拟 API 的变化.模拟 API 中的更改可能会使应用验证变红，从而触发应用实现的更改.这样您就不会错过外部 API 和应用实现之间的任何差距(理想情况下).

If along the way, external API changes, API verification may turn red, triggering changes in mock API. Changes in mock API may make app verification turn red, triggering changes in app implementation. This way you never miss any gap between external API and app implementation (ideally).

拥有模拟 API + API 验证的另一个额外好处是，您的开发人员可以将其用作 API 应该如何工作的文档/规范.

Another extra benefit of having a mock API + API verification is that your developers can use it as a documentation/specification of how the API is supposed to work.

这篇关于集成测试:模拟外部 API 与使用外部 API 沙箱的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！