覆盖率与可达代码 [英] Coverage vs reachable code

问题描述

问:如何检测真实的测试覆盖率?

Q: how to detect real test coverage ?

我注意到代码覆盖率指标和测试质量存在一个问题:100％的代码覆盖率并不意味着代码已经过真正的测试.

I've noticed one problem with code coverage metric and test quality: 100% code coverage doesn't mean that code is really tested.

有时候，即使测试没有涵盖所有内容，它也会提供100％的覆盖率.问题在于覆盖范围的定义，我们假设 coverage ==可达代码.

Sometimes test gives 100% coverage even that it doesn't cover everything. Problem lays in coverage definition, we assume coverage==reachable code.

但是事实并非如此，代码可以100％可达，但测试中没有100％覆盖.

But it's not true, code could be 100% reachable but not 100% covered with the test.

以示例为例，该测试提供100％的覆盖率(EMMA)，但实际上它不覆盖将传递给服务模拟的值.因此，如果值将被更改，则测试不会失败.

Take a look into example, this test gives 100% coverage (EMMA), but in reality it doesn't cover values which will be passed to service mock. So, if value will be changed, test won't fail.

示例:

public class User {
  public static final int INT_VALUE = 1;
  public static final boolean BOOLEAN_VALUE = false;
  public static final String STRING_VALUE = "";
  private Service service;

  public void setService(Service service) {
      this.service = service;
  }

  public String userMethod() {
      return service.doSomething(INT_VALUE, BOOLEAN_VALUE, STRING_VALUE);
  }
}

并对其进行测试:

public class UserTest {

  private User user;
  private Service easyMockNiceMock;

  @Before
  public void setUp() throws Exception {
      user = new User();
      easyMockNiceMock = EasyMock.createNiceMock(Service.class);
  }

  @Test
  public void nonCoverage() throws Exception {
      // given
      user.setService(easyMockNiceMock);
      expect(easyMockNiceMock.doSomething(anyInt(), anyBoolean(), (String) anyObject())).andReturn("");
      replay(easyMockNiceMock);
      // when
      user.userMethod();
      // then
      verify(easyMockNiceMock);
  }
}

推荐答案

看看 Jester ，执行变异测试.从该站点:

Take a look at Jester, which performs mutation testing. From the site:

Jester查找测试未涵盖的代码.小丑弄了一些 更改代码，运行测试，以及测试是否通过Jester 显示一条消息，说明发生了什么变化.小丑包括脚本 用于生成显示所做更改的网页，这些更改并未引起 测试失败.

Jester finds code that is not covered by tests. Jester makes some change to your code, runs your tests, and if the tests pass Jester displays a message saying what it changed. Jester includes a script for generating web pages that show the changes made that did not cause the tests to fail.

Jester与代码覆盖工具不同，因为它可以找到代码 通过运行测试执行但未实际测试. 小丑的方法称为变异测试或自动错误 播种.但是，Jester并不能替代代码 覆盖工具，只是作为一种补充方法.

Jester is different than code coverage tools, because it can find code that is executed by the running of tests but not actually tested. Jester's approach is called mutation testing or automated error seeding. However, Jester is not meant as a replacement for code coverage tools, merely as a complementary approach.

这篇关于覆盖率与可达代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！