CMake-作为构建过程的一部分运行测试并将stdout输出捕获到文件 [英] CMake - run test as part of the build process and capture stdout output to file

问题描述

我们有几个单元测试，我们希望在构建过程中运行它们。

We have several unit tests which we would like to be run as part of our build process.

要实现这一点，我有一个帮助程序脚本，可创建自定义命令运行测试，如果成功，则创建文件 test_name.passed 。

To achieve this I have a helper script which creates a custom command that runs the test, and if successful, creates a file "test_name.passed".

我然后添加一个自定义目标 test_name.run 取决于 test_name.passed 。

I then add a custom target "test_name.run" which depends on "test_name.passed".

这个想法是，如果 test_name.passed 不存在或早于 test_name ，则将运行自定义命令。

The idea is that if "test_name.passed" doesn't exist or is older than "test_name", the custom command will be run.

内部版本将继续运行自定义命令，直到测试通过。一旦通过，后续的构建将不会调用自定义命令，因此该测试将在不需要时运行。

Builds will continue to run the custom command until the test passes. Once it passes, subsequent builds won't call the custom command, so the test won't be run when it doesn't need to.

到目前为止

以下是所有脚本的工作原理：

Here is the script:

# create command which runs the test and creates a sentinel file if it passes
add_custom_command(
    OUTPUT  ${TEST_NAME}.passed
    COMMAND $<TARGET_FILE:${TEST_NAME}>
    COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
    DEPENDS ${TEST_NAME}
    )

# create test.run module which depends on test.passed
add_custom_target(${TEST_NAME}.run
    ALL
    DEPENDS ${TEST_NAME}.passed
    )

问题- stdout

The problem - noise on stdout

问题在于我们的测试通常会将一吨信息记录到 stdout 中，这会使构建过程非常嘈杂。

The problem is that our tests often log a tonne of information to stdout, and it makes for a very noisy build.

我正在尝试捕获 stdout 到文件，只有在失败的情况下，才显示测试输出。

I'm trying to now capture stdout to a file, and only in the event of a failure, display the test output.

我的第一次尝试是尝试使用Bash Shell脚本语法-将 stdout 捕获到文件中，然后在退出状态为错误时，将文件保存为空。

My first attempt was to try a Bash shell scripting syntax - capture stdout into a file and when the exit status is an error, cat the file.

add_custom_command(
    OUTPUT  ${TEST_NAME}.passed
    COMMAND $<TARGET_FILE:${TEST_NAME}> > ${TEST_NAME}.output || cat ${TEST_NAME}.output
    COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
    DEPENDS ${TEST_NAME}
    )

这是行不通的，即使测试失败，我也会收到哨兵 test_name.passed 文件已创建，这意味着我下次尝试构建该文件时，它认为测试已通过。

This doesn't work, as even if the test fails I am getting the sentinal "test_name.passed" file created, which means the next time I try to build it thinks the test passed.

可能的不合标准的修补程序

通过与 ctest 集成，我可以通过ctest运行每个测试，并使用命令行选项-故障输出

By integrating with ctest, I can run each test through ctest and use the command-line option --output-on-failure

add_custom_command(
    OUTPUT  ${TEST_NAME}.passed
    COMMAND ctest --build-config $<CONFIGURATION> --tests-regex ${TEST_NAME} --output-on-failure
    COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
    DEPENDS ${TEST_NAME}
    )

问题在于两倍。

1. 极大地增加了构建时间。每个测试都必须通过单独的ctest流程执行，所有注册的测试名称都将根据正则表达式进行解析，依此类推。随着单个测试数量的增加，我们的额外时间大大增加了。


2. ctest默认会输出很多噪音。指定-quiet 标志会抑制-failure-output 标志，因此您可以产生嘈杂的输出或没有输出-没有办法只能解决失败。

1. It vastly increases the build time. Each and every test has to be executed through a separate ctest process, all the registered test names parsed against the regex, etc. With the number of individual tests we have the extra time adds up considerably.
2. ctest outputs a lot of noise by default. Specifying the --quiet flag suppresses the --output-on-failure flag, so you can either have noisy output or no output - there is no way to only get the failures.

问题

有没有办法实现我想要的？

Is there a way to achieve what I want?

即：

• 手动运行测试（即：不通过ctest）


• 将输出捕获到文件中


• 仅在测试退出状态指示失败时才输出该文件。


• 如果测试退出状态指示成功，则触摸哨兵文件。

• Run the test manually (ie: not through ctest)
• Capture the output to a file
• Only output that file in the event of the test exit status indicating failure.
• Touch a sentinel file in the event of the test exit status indicating success.

奖金指向跨平台方法，但是如果必须仅是Linux，那就这样吧。

Bonus points for a cross-platform method, but if it has to be Linux only, then so be it.

推荐答案

回答我自己的问题问题，因为我找到了适合我的解决方案。

Answering my own question because I found a solution which works for me.

在下面的摘录中， $ {ARG_NAME} 是我们已经构建并要运行的测试二进制文件。

In the below excerpt, ${ARG_NAME} is the test binary which we've built and want to run.

运行测试时创建了2个文件

There are 2 files created whilst running the tests

• $ {ARG_NAME} .output ，其中包含测试输出


• $ {ARG_NAME} .passed 是通过测试成功创建的哨兵文件

• ${ARG_NAME}.output which contains the test output
• ${ARG_NAME}.passed which is a sentinel file created if the test is successful

custom_command 由几个命令组成，我们依靠的事实是，一旦其中一个失败，cmake就会停止执行。

The custom_command consists of several commands, and we rely on the fact that cmake will stop executing as soon as one of them fails.

我们运行测试：

${ARG_NAME}

所有测试输出都重定向到 $ {ARG_NAME} .output ，因此在测试通过时我们不会污染stdout

All test output is redirected to ${ARG_NAME}.output, so we don't pollute stdout when the tests pass

${ARG_NAME} >> ${OUTPUT_FILE} 2>&1

在失败的情况下使用测试的退出状态我们对文件进行分类：

Using the test's exit status, in the event of a failure we cat the file:

${ARG_NAME} >> ${OUTPUT_FILE} 2>&1 || cat ${OUTPUT_FILE}

但是，只有在测试通过后，我们才想创建前哨文件，因此我们都将文件放在目录中并运行 false

However, only when the test passes do we want to create the sentinel file, so we both cat the file and run false

${ARG_NAME} >> ${OUTPUT_FILE} 2>&1 || (cat ${OUTPUT_FILE} && false)

如果测试通过（退出状态为0 ），那么我们就不会运行 cat输出&& false ，因此运行下一个cmake命令

If the test passes (exit status is 0), then we won't have run the cat output && false, so then the next cmake command is run

${CMAKE_COMMAND} -E touch ${PASSED_FILE}

最后，我们创建一个 custom_target 取决于 .passed 文件

Finally we create a custom_target which depends on the .passed file

set(OUTPUT_FILE ${CMAKE_CURRENT_BINARY_DIR}/${ARG_NAME}.output)
set(PASSED_FILE ${CMAKE_CURRENT_BINARY_DIR}/${ARG_NAME}.passed)

# create test.passed command which runs this test and creates a sentinel file
# if it passes
add_custom_command(
    OUTPUT
        ${PASSED_FILE}

    COMMAND
        ${CMAKE_CURRENT_BINARY_DIR}/${ARG_NAME} >> ${OUTPUT_FILE} 2>&1 
              || (cat ${OUTPUT_FILE} && false)

    COMMAND
        ${CMAKE_COMMAND} -E touch ${PASSED_FILE}

    COMMENT
        "Running ${ARG_NAME} tests"

    DEPENDS
        ${ARG_NAME}

    USES_TERMINAL
    )

# create test.run target which depends on test.passed
add_custom_target(${ARG_NAME}.run
    ALL
    DEPENDS ${PASSED_FILE}
    )

这篇关于CMake-作为构建过程的一部分运行测试并将stdout输出捕获到文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！