LCOV/GCOV分支覆盖范围遍布整个C ++生产分支 [英] LCOV/GCOV branch coverage with C++ producing branches all over the place

问题描述

我们正在使用LCOV/GCOV来对我们的项目进行测试覆盖.最近，我们尝试另外启用分支覆盖.但这看起来并不能产生我们从高级开发人员的角度所期望的结果.

We are using LCOV/GCOV to produce test coverage of our projects. Recently we tried to enable branch-coverage additionally. But it looks like, this just doesn't yield the results we expected from a high-level developer view.

在C ++中使用branch-coverage可以使报表随处可见.我们怀疑(正如对问题的搜索所示)，大多数情况下，异常处理代码会创建这些隐藏的分支".而且GCOV/LCOV似乎没有跳过这些.

Using branch-coverage with C++ blows the report up with branches all over the place. We suspect (as the searching for the issues indicates) that mostly exception handling code creates these "hidden branches". And GCOV/LCOV doesn't seem to skip over these.

我创建了一个小测试项目来显示问题: https://github.com/ghandmann/lcov-branch-coverage-怪异

I created a small test project to show the problem: https://github.com/ghandmann/lcov-branch-coverage-weirdness

当前，我们使用Ubuntu 16.04.与:

Currently we use Ubuntu 16.04. with:

• gcc v5.4
• lcov& genhtml v1.12

我们的生产代码是在启用c ++ 11的情况下构建的.最小的示例不是在启用c ++ 11的情况下构建的，但是当我们对所有不同的选项(c ++标准，优化，-fno-exceptions)进行了一些实验时，我们没有得出合格的结果.

Our production code is built with c++11 enabled. The minimal example isn't built with c++11 enabled, but as we experimented a bit with all different options (c++ standard, optimization, -fno-exceptions) we didn't come up with a passable result.

有人有想法吗?小费?我们使用任何错误的方式吗?如其他地方所述，这真的是预期的行为吗?

Anybody got some ideas? Tipps? Are we using anything the wrong way? Is this - as stated somewhere else - really expected behavior?

更新:

gcc帮助邮件列表中也指出，这些隐藏的分支"是由于异常处理而发生的.因此，将-fno-exceptions开关添加到gcc可以为简单"程序提供100％的分支覆盖率.但是当禁用异常时，gcc拒绝编译实际使用异常的代码(例如try-catch，throw).因此，对于实际的生产代码，这不是一个选择.看起来，在这种情况下，您只需声明〜50％的覆盖率就是新的100％. ;)

As also pointed out on the gcc-help mailing list, these "hidden branches" occur because of exception handling. So adding the -fno-exceptions switch to gcc produces 100% branch coverage for "simple" programs. But when exceptions are disabled, gcc refuses to compile code which actually uses exceptions (e.g. try-catch, throw). Therefore for real production code this is not an option. Looks like, you have to simply declare ~50% coverage to be the new 100% in this case. ;)

推荐答案

问题在于，GCC还会记录可能由于某些抛出异常而导致范围退出的每行的分支信息(例如，在具有GCC 6.3.1的Fedora 25上)和lcov 1.12).

The thing is that GCC also records branch information for each line where a scope exit due to some thrown exception is possible (e.g. on Fedora 25 with GCC 6.3.1 and lcov 1.12).

此信息的值是有限的.分支覆盖率数据的主要用例是复杂的if语句，该语句具有多子句逻辑表达式，如下所示:

The value of this information is limited. The main use case for branch coverage data are complicated if-statements that have a multi-clausal logical expression like this:

if (foo < 1 && (bar > x || y == 0))

说您有兴趣验证您的测试套件是否也涵盖了bar > x案例，或者您只是拥有y == 0的测试案例.

Say you are interested to verify whether your test suite also covers the bar > x case or if you just have test cases where y == 0.

为此，lcov的genhtml的分支覆盖范围数据收集和可视化非常有用.对于简单的if语句，如

For this, branch coverage data collection and the visualization by lcov's genhtml is useful. For simple if-statements like

if (p == nullptr) {
  return false;
}
return true;

您不需要分支机构的覆盖率数据，因为您可以通过查看以下各行的覆盖率来查看分支机构是否已被采用.

you don't need branch coverage data because you see whether the branch was taken or not via looking at the coverage of the following lines.

由lcov生成的genhtml的输入采用相对简单的文本格式(请参见 filterbr.py .另见lcov/genhtml，另请参见 gen-coverage.py 处理步骤和示例项目，将生成的跟踪文件上传到codecov (codecov不使用genhtml，但可以导入lcov跟踪文件并显示分支覆盖率数据.

The input of genhtml that is generated by lcov is in a relatively simple text format (cf. geninfo(1)). Thus, you can post-process it such that all lines that start with BRDA: and don't belong to an if-statement are removed. See for example filterbr.py which implements this approach. See also gen-coverage.py for the other lcov/genhtml processing steps and an example project where the resulting trace file is uploaded to codecov (codecov doesn't use genhtml but can import lcov trace files and displays branch coverage data).

• 仅当您的C ++代码不使用任何例外时，才可以选择禁用例外
• 使用类似-O1 -fno-omit-frame-pointer -fno-optimize-sibling-calls的内容进行编译可以稍微减少已记录的分支覆盖数据的数量，但数量不多
• Clang支持GCOV样式覆盖率收集，但也实现了另一种方法，称为'基于源代码的覆盖率(使用-fprofile-instr-generate -fcoverage-mapping进行编译，并使用llvm-profdata和llvm-cov进行后处理).不过，该工具链不支持分支机构覆盖率数据(截至2017-05-01).
• 默认情况下，lcov + genhtml不会生成分支覆盖率数据-有时您确实不需要(见上文)，因此可以选择禁用它(请参阅--rc lcov_branch_coverage=0和--no-branch-coverage )

• disabling exceptions is only an option when your C++ code doesn't use any
• compiling with something like -O1 -fno-omit-frame-pointer -fno-optimize-sibling-calls somewhat reduces the number of recorded branch coverage data, but not much
• Clang supports GCOV style coverage collection but also implements a different approach, called 'source-based code coverage' (compile with -fprofile-instr-generate -fcoverage-mapping and post-process with llvm-profdata and llvm-cov). That toolchain doesn't support branch coverage data, though (as of 2017-05-01).
• by default, lcov+genhtml don't generate branch coverage data - sometimes you don't really need it (see above), thus, it is then an option to disable it (cf. --rc lcov_branch_coverage=0 and --no-branch-coverage)

这篇关于LCOV/GCOV分支覆盖范围遍布整个C ++生产分支的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！