如何适应我的单元测试cmake和ctest？ [英] How to adapt my unit tests to cmake and ctest?

问题描述

到目前为止，我使用了一个临时单元测试程序 - 基本上是一个由批处理文件自动运行的单元测试程序。虽然很多这些明确检查他们的结果，更多的骗子 - 他们将结果转储到文本文件的版本。测试结果的任何变化都被subversion标记，我可以很容易地识别出变化是什么。许多测试输出点文件或一些其他形式，让我得到一个视觉表示的输出。

Until now, I've used an improvised unit testing procedure - basically a whole load of unit test programs run automatically by a batch file. Although a lot of these explicitly check their results, a lot more cheat - they dump out results to text files which are versioned. Any change in the test results gets flagged by subversion and I can easily identify what the change was. Many of the tests output dot files or some other form that allows me to get a visual representation of the output.

麻烦的是，我切换到使用cmake。使用cmake流意味着使用源代码外部构建，这意味着转储结果在共享源/构建文件夹中的方便，并将它们与源一起进行版本化并不能实现。

The trouble is that I'm switching to using cmake. Going with the cmake flow means using out-of-source builds, which means that convenience of dumping results out in a shared source/build folder and versioning them along with the source doesn't really work.

作为替代，我会像做的是告诉单元测试工具在哪里找到预期结果的文件（在源树中），并让它做比较。如果失败，它应该提供实际的结果和差异列表。

As a replacement, what I'd like to do is to tell the unit test tool where to find files of expected results (in the source tree) and get it to do the comparison. On failure, it should provide the actual results and diff listings.

这是可能的，还是应该采取完全不同的方法？

Is this possible, or should I take a completely different approach?

很显然，我可以忽略ctest，只是适应我一直对源代码构建做的。我可以版本我的文件夹，在那里的所有的构建生活，例如（当然使用'忽略'）。这是理智吗？可能不是，因为每个构建最终会得到预期结果的单独副本。

Obviously, I could ignore ctest and just adapt what I've always done to out-of-source builds. I could version my folder-where-all-the-builds-live, for instance (with liberal use of 'ignore' of course). Is that sane? Probably not, as each build would end up with a separate copy of the expected results.

此外，任何建议的方式使用cmake / ctest gratefuly收到。我用cmake浪费了一点时间，不是因为它很糟糕，而是因为我不明白如何最好地使用它。

Also, any advice on the recommended way to do unit testing with cmake/ctest gratefuly received. I wasted a fair bit of time with cmake, not because it's bad, but because I didn't understand how best to work with it.

EDIT

EDIT

最后，我决定保持cmake / ctest单元测试端尽可能简单。为了测试实际的预期结果，我发现我的库中有以下函数的家...

In the end, I decided to keep the cmake/ctest side of the unit testing as simple as possible. To test actual against expected results, I found a home for the following function in my library...

bool Check_Results (std::ostream              &p_Stream  ,
                    const char                *p_Title   ,
                    const char               **p_Expected,
                    const std::ostringstream  &p_Actual   )
{
  std::ostringstream l_Expected_Stream;

  while (*p_Expected != 0)
  {
    l_Expected_Stream << (*p_Expected) << std::endl;
    p_Expected++;
  }

  std::string l_Expected (l_Expected_Stream.str ());
  std::string l_Actual   (p_Actual.str ());

  bool l_Pass = (l_Actual == l_Expected);

  p_Stream << "Test: " << p_Title << " : ";

  if (l_Pass)
  {
    p_Stream << "Pass" << std::endl;
  }
  else
  {
    p_Stream << "*** FAIL ***" << std::endl;
    p_Stream << "===============================================================================" << std::endl;
    p_Stream << "Expected Results For: " << p_Title << std::endl;
    p_Stream << "-------------------------------------------------------------------------------" << std::endl;
    p_Stream << l_Expected;
    p_Stream << "===============================================================================" << std::endl;
    p_Stream << "Actual Results For: " << p_Title << std::endl;
    p_Stream << "-------------------------------------------------------------------------------" << std::endl;
    p_Stream << l_Actual;
    p_Stream << "===============================================================================" << std::endl;
  }

  return l_Pass;
}

典型的单元测试现在看起来像...

A typical unit test now looks something like...

bool Test0001 ()
{
  std::ostringstream l_Actual;

  const char* l_Expected [] =
  {
    "Some",
    "Expected",
    "Results",
    0
  };

  l_Actual << "Some" << std::endl
           << "Actual" << std::endl
           << "Results" << std::endl;

  return Check_Results (std::cout, "0001 - not a sane test", l_Expected, l_Actual);
}

我需要一个可重复使用的数据转储函数， std :: ostream& 类型，因此可以转储到实际结果流。

Where I need a re-usable data-dumping function, it takes a parameter of type std::ostream&, so it can dump to an actual-results stream.

推荐答案

我会使用CMake的独立脚本模式来运行测试和比较输出。通常对于单元测试程序，您可以写 add_test（testname testexecutable），但您可以运行任何命令作为测试。

I'd use CMake's standalone scripting mode to run the tests and compare the outputs. Normally for a unit test program, you would write add_test(testname testexecutable), but you may run any command as a test.

如果你写一个脚本runtest.cmake并通过这个执行你的单元测试程序，那么runtest.cmake脚本可以做任何它喜欢 - 包括使用 cmake -E compare_files 实用程序。在CMakeLists.txt文件中需要类似以下内容：

If you write a script "runtest.cmake" and execute your unit test program via this, then the runtest.cmake script can do anything it likes - including using the cmake -E compare_files utility. You want something like the following in your CMakeLists.txt file:

enable_testing()
add_executable(testprog main.c)
add_test(NAME runtestprog
    COMMAND ${CMAKE_COMMAND}
    -DTEST_PROG=$<TARGET_FILE:testprog>
    -DSOURCEDIR=${CMAKE_CURRENT_SOURCE_DIR}
    -P ${CMAKE_CURRENT_SOURCE_DIR}/runtest.cmake)

这将运行一个脚本（cmake -P runtest.cmake）定义2个变量：TEST_PROG，设置为测试可执行文件的路径，SOURCEDIR，设置为当前源目录。你需要第一个知道运行哪个程序，第二个知道在哪里找到预期的测试结果文件。 runtest.cmake 的内容是：

This runs a script (cmake -P runtest.cmake) and defines 2 variables: TEST_PROG, set to the path of the test executable, and SOURCEDIR, set to the current source directory. You need the first to know which program to run, the second to know where to find the expected test result files. The contents of runtest.cmake would be:

execute_process(COMMAND ${TEST_PROG}
                RESULT_VARIABLE HAD_ERROR)
if(HAD_ERROR)
    message(FATAL_ERROR "Test failed")
endif()

execute_process(COMMAND ${CMAKE_COMMAND} -E compare_files
    output.txt ${SOURCEDIR}/expected.txt
    RESULT_VARIABLE DIFFERENT)
if(DIFFERENT)
    message(FATAL_ERROR "Test failed - files differ")
endif()

第一个 execute_process 运行测试程序，这将写出output.txt。如果有效，那么下一个 execute_process 有效地运行 cmake -E compare_files output.txt expected.txt 。文件expected.txt是源树中已知的良好结果。如果有差异，它会错误输出，因此您可以看到失败的测试。

The first execute_process runs the test program, which will write out "output.txt". If that works, then the next execute_process effectively runs cmake -E compare_files output.txt expected.txt. The file "expected.txt" is the known good result in your source tree. If there are differences, it errors out so you can see the failed test.

这不是打印出差异; CMake没有完全隐藏的diff实现。现在你使用Subversion来查看行是否发生了变化，所以一个明显的解决方案是将最后一部分更改为：

What this doesn't do is print out the differences; CMake doesn't have a full "diff" implementation hidden away within it. At the moment you use Subversion to see what lines have changed, so an obvious solution is to change the last part to:

if(DIFFERENT)
    configure_file(output.txt ${SOURCEDIR}/expected.txt COPYONLY)
    execute_process(COMMAND svn diff ${SOURCEDIR}/expected.txt)
    message(FATAL_ERROR "Test failed - files differ")
endif()

这将覆盖源代码树on失败然后运行svn diff就可以了。问题是你不应该真的去这样改变源代码树。当你第二次运行测试，它通过！一个更好的方法是安装一些视觉差异工具，并对你的输出和预期的文件运行。

This overwrites the source tree with the build output on failure then runs svn diff on it. The problem is that you shouldn't really go changing the source tree in this way. When you run the test a second time, it passes! A better way is to install some visual diff tool and run that on your output and expected file.

这篇关于如何适应我的单元测试cmake和ctest？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！