如何使用 cmake 测试预期会因异常而失败的进程?(例如，由于 clang 的地址清理程序导致的失败) [英] How can I use cmake to test processes that are expected to fail with an exception? (e.g., failures due to clang's address sanitizer)

问题描述

我有一些测试可以测试 clang 的地址清理程序是否捕获特定错误.(我想确保我对它可以捕获的错误类型的理解是正确的，并且未来的版本会继续捕获我期望它们捕获的错误类型.)这意味着我有几个测试失败了OTHER_FAULT，这似乎是 clang 的运行时报告错误的固定方式.

I've got some tests that test that clang's address sanitizer catch particular errors. (I want to ensure my understanding of the types of error it can catch is correct, and that future versions continue to catch the type of errors I'm expecting them to.) This means I have several tests that fail by crapping out with an OTHER_FAULT, which appears to be the fixed way that clang's runtime reports an error.

我已将这些测试的 WILL_FAIL 标志设置为 TRUE，但这似乎只是检查成功、无异常失败的返回值.如果进程因异常而终止，cmake 仍将其归类为失败.

I've set the WILL_FAIL flag to TRUE for these tests, but this only seems to check the return value from a successful, exception-free failure. If the process terminates with an exception, cmake still classes it as a failure.

我也尝试使用 PASS_REGULAR_EXPRESSION 来观察发生此错误时打印出的区别消息，但同样，如果测试因异常终止，cmake 似乎将测试归类为失败.

I've also tried using PASS_REGULAR_EXPRESSION to watch for the distinguishing messages that are printed out when this error occurs, but again, cmake seems to class the test as a failure if it terminates with an exception.

我能做些什么来解决这个问题吗?

Is there anything I can do to get around this?

(特定于clang的答案也是一种选择！-但我怀疑这将是我最后一次需要测试这样的东西，所以如果可能的话，我更愿意知道如何使用cmake进行测试)

(clang-specific answers are also an option! - but I doubt this will be the last time I need to test something like this, so I'd prefer to know how to do it with cmake generally, if it's possible)

推荐答案

CTest 只为测试程序的结果提供基本的、常用的解释器.为了实现其他解释器，您可以编写简单的程序/脚本，它包装测试并根据需要解释其结果.例如.C 程序(适用于 Linux):

CTest provides only basic, commonly used interpretators for result of test programs. For implement other interpretators you can write simple program/script, which wraps the test and interpret its result as needed. E.g. C program (for Linux):

test_that_crash.c:

#include <unistd.h>
#include <stdlib.h>
#include <sys/types.h>

int main(int argc, char** argv)
{
    pid_t pid = fork();
    if(pid == -1)
    {
        // fork fails
        return 1;
    }
    else if(pid)
    {
        // Parent - wait child and interpret its result
        int status = 0;
        wait(&status);
        if(WIFSIGNALED(status)) return 0; // Signal-terminated means success
        else return 1;
    }
    else
    {
        // Child - execute wrapped command
        execvp(argv[1], argv + 1);
        exit(1);
    }
}

这个程序可以在CMake中使用如下:

This program can be used in CMake as follows:

CMakeLists.txt:

# Compile our wrapper
add_executable(test_that_crash test_that_crash.c)
# Similar to add_test(name command), but test is assumed successfull only if it is crashed(signalled)
macro(add_test_crashed name command)
    # Use generic flow of add_test() command for automatically recognize our executable target
    add_test(NAME ${name} COMMAND test_that_crash ${command} ${ARGN})
endmacro(add_test_crashed)
# ...

# Add some test, which should crash
add_test_crashed(clang.crash.1 <clang-executable> <clang-args>)

这篇关于如何使用 cmake 测试预期会因异常而失败的进程?(例如，由于 clang 的地址清理程序导致的失败)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！