如何在使用单个测试项目时将所有QtTestLib单元测试的结果组合到一个文件中？ [英] How to compose all QtTestLib unit tests' results in a single file while using a single test project?

问题描述

在我们的项目中，我们使用QtTestLib进行单元测试。原因是整个项目已经尽可能使用Qt并且它是一个GUI应用程序，因此我们希望能够测试GUI界面。

In our project we are using the QtTestLib for a unit testing. The reasons are that the whole project already uses Qt whenever it's possible and it's a GUI application, so we wanted to have ability for testing GUI interfaces.

我们的项目由MSVC，因此我们不希望每个测试都有单独的项目文件，因为它会使解决方案混乱。因此，我们为所有测试创建了一个项目。所有测试都应该在CIS（连续集成）上实现自动化，因此我们尝试使用XSLT转换通过XML格式的输出文件将测试插入到Hudson。

Our project is compiled by the MSVC, so we didn't want to have a separated project file for each test 'coz it will clutter the solution. Thus we have created a single project for all tests. All testing should be automated on a CIS (continuous integration), so we tried to plug our tests to the Hudson through an output file in XML format using some XSLT transformations.

但是似乎测试输出存在问题。如果您对所有测试使用单个main（），而仅将cmd行参数传递给每个测试：

But it seems there is a problem with tests' output. If you use a single main() for all tests and merely transmit cmd line arguments to each test:

#include "MyFirstTest.h"
#include "MySecondTest.h"

int main(int argc, char **argv)
{
  int result = 0;
  MyFirstTest test1;
  result |= QTest::qExec(&test1, argc, argv);
  MySecondTest test2;
  result |= QTest::qExec(&test2, argc, argv);
  return result;
}

然后，您将获得多次重写的结果文件。因此，如果您想使用输出文件（例如xml）对其进行某种程度的自动化，则只会得到最后的结果。其他所有内容都会被覆盖。

then you will get a result file rewrited multiple times. So if you want to automate it somewhat using output file (xml for example), you'll get only the last result in it. All other will be overwritten.

我们已经尝试过这种方法，它不能让您使用某些连续的集成系统，例如Hudson。所以我的问题是：是否有机会将结果附加到一个输出文件中？当然，我们可以使用一些变通办法，例如通过QTest :: qExec（）使用修改后的参数运行每个测试，以将结果写入单独的文件中，但这似乎不是最好的方法。理想情况下，我希望有一个单一的结果文件以与CIS一起使用。

We have tried that approach already, it does not give you ability to use some continuous integration systems like Hudson. So my question will be: is there any opportunities to append results in one output file? Of course we can use some workarounds like running each test by QTest::qExec() with a modified parameters to write results in separate files, but it doesn't seem to be the best way. Ideally I want to have a single result file to use it with CIS.

推荐答案

使用此技巧，您可以收集单个测试xml向临时缓冲区/文件报告；全部来自单个测试二进制文件。让我们使用QProcess从一个二进制文件中收集单独的测试输出；测试使用修改后的参数调用自身。首先，我们引入一个特殊的命令行参数，该参数可以充分利用子测试-所有这些仍然在您的测试可执行文件中。为了方便起见，我们使用了接受QStringList的重载qExec函数。然后，我们可以插入/删除 -subtest

With this trick you can collect the individual test xml reports to temporary buffers/files; all from a single test binary. Lets employ QProcess to collect separate test outputs from within one binary; the test calls itself with modified arguments. First, we introduce a special command-line argument that harnesses the subtests proper - all still within your test executable. For our convenience, we use the overloaded qExec function that accepts a QStringList. Then we can insert/remove our "-subtest" argument more easily.

// Source code of "Test"

int
main( int argc, char** argv )
{
  int result = 0;

  // The trick is to remove that argument before qExec can see it; As qExec could be
  // picky about an unknown argument, we have to filter the helper 
  // argument (below called -subtest) from argc/argc; 

  QStringList args;

  for( int i=0; i < argc; i++ )
  {
     args << argv[i];
  }

  // Only call tests when -subtest argument is given; that will usually
  // only happen through callSubtestAndStoreStdout

  // find and filter our -subtest argument

  size_t pos = args.indexOf( "-subtest" );

  QString subtestName;

  if( (-1 != pos) && (pos + 1 < args.length()) )
  {
    subtestName = args.at( pos+1 );

    // remove our special arg, as qExec likely confuses them with test methods

    args.removeAt( pos );
    args.removeAt( pos );

    if( subtestName == "test1" )
    {
      MyFirstTest test1;
      result |= QTest::qExec(&test1, args);
    }

    if( subtestName == "test2" )
    {
      MySecondTest test2;
      result |= QTest::qExec(&test2, args);
    }

    return result;
}

然后，在您的脚本/命令行调用中：

Then, in your script/commandline call:

./Test -subtest test1 -xml ... >test1.xml
./Test -subtest test2 -xml ... >test2.xml

在这里-我们有办法分离测试输出。现在，我们可以继续使用QProcess的功能来为您收集标准输出。只需将这些行添加到主行即可。这个想法是，如果不需要显式测试，而是使用特殊参数，则再次调用我们的可执行文件：

and here you are - we have the means to separate the tests output. Now we can continue to use QProcess'es ability to collect stdout for you. Just append these lines to your main. The idea is to call our executable again, if no explicit tests are requested, but with our special argument:

bool
callSubtestAndStoreStdout(const String& subtestId, const String& fileNameTestXml, QStringList args)
{
   QProcess proc;

   args.pop_front();

   args.push_front( subtestId );
   args.push_front( "-subtest" );

   proc.setStandardOutputFile( fileNameTestXml );

   proc.start( "./Test", args );

   return proc.waitForFinished( 30000 ); // int msecs
}

int 
main( int argc, char** argv )
{
   .. copy code from main in box above..

   callSubtestAndStoreStdout("test1", "test1.xml", args);
   callSubtestAndStoreStdout("test2", "test2.xml", args);

   // ie. insert your code here to join the xml files to a single report

   return result;
}

然后在脚本/命令行调用中：

Then in your script/commandline call:

./Test -xml           # will generate test1.xml, test2.xml

希望将来的QTestLib版本使此操作更容易。

Indeed, hopefully future QTestLib versions makes this easier to do.

这篇关于如何在使用单个测试项目时将所有QtTestLib单元测试的结果组合到一个文件中？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！