cucumber.json报告被重新运行场景报告覆盖 [英] cucumber.json report getting overwritten by rerun scenario report
问题描述
我有UI测试项目和具有相同技术堆栈的API测试项目（JAVA1.8，Cucumber-JVM，JUnit，Maven），这两个项目都向我展示了这个问题。可能是因为两者都存在相同的依赖关系。
I have got UI Test project and a API test project with same technology stack (JAVA1.8, Cucumber-JVM, JUnit, Maven) and both projects are showing me this problem. Probably because same set of dependencies are present in both.
我使用了maven-surefire-plugin内置功能< groupId> io.cucumber< / groupId> 添加了黄瓜依赖项，而不是< groupId> info.cukes< /的groupId> 。这两个都有自己的cucumber-java和cucumber-jvm依赖版本。
I have employed the Flaky test re-run mechanism using maven-surefire-plugin build-in functionality <rerunFailingTestsCount>1</rerunFailingTestsCount>. Also, I have cucumber dependencies added based on <groupId>io.cucumber</groupId> and not <groupId>info.cukes</groupId>. Both these have their own version of cucumber-java and cucumber-jvm dependencies.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<configuration>
<rerunFailingTestsCount>1</rerunFailingTestsCount>
</configuration>
<executions>
<execution>
<id>acceptance-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<forkCount>1</forkCount>
<reuseForks>true</reuseForks>
<includes>
<include>**/*Runner.class</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-spring</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-jvm</artifactId>
<version>2.4.0</version>
<type>pom</type>
</dependency>
只有RUNNER文件代码
ONLY RUNNER FILE CODE
@RunWith(Cucumber.class)
@ContextConfiguration(locations = {"file:/src/test/resources/spring-config.xml"})
@CucumberOptions(
glue = "com.test.uitest",
features = "classpath:cucumber",
tags = {"~@ignore","@ui_home"},
monochrome = true,
plugin = {"pretty", "html:target/cucumber-reports",
"json:target/cucumber-reports/cucumber.json",
"rerun:target/rerun.txt"} //Creates a text file with failed scenarios
)
public class AllTestsRunner {
}
现在很明显，我需要让另一名参赛者使用以下代码（根据StackOverflow上的其他论坛和帖子）
Now Aparently, I need to have another runner with following code in it (as per other forums and threads here on StackOverflow)
@RunWith(Cucumber.class)
@CucumberOptions(
monochrome = true,
glue = "com.test.uitest",
features = "@target/rerun.txt", //Cucumber picks the failed scenarios from this file
format = {"pretty", "html:target/rerun-reports",
"json:target/cucumber-reports/rerun-report.json"}
)
public class FailedTestsRunner {
}
但我不需要这个第二名选手，因为重新运行机制绝对精彩，只有顶级赛跑者。甚至，也不需要在1st runner中生成rerun.txt文件.maven-surefire插件（v_2.21.0）中的内置机制以及io.cucumber v_2.4.0可以正常工作，如果在第一次执行期间任何场景失败，它会自动重新运行而不会在rerun.txt文件中记录它。
But I don't need to have this 2nd runner, as the rerun mechanism works absolutely brilliantly with just the one runner on top. Even, there is no need of rerun.txt file generated in 1st runner.The in-build mechanism within maven-surefire plugin (v_2.21.0) along with io.cucumber v_2.4.0 works perfectly and if any scenario fails during 1st execution, it reruns automatically without recording it inside rerun.txt file.
我的功能文件中有5个场景。如果所有这些都在第一轮中通过。它使用cucumber.json报告成功生成报告，显示所有5个场景。
但是，如果（比方说）5个场景中有2个失败，并且这些场景在重新运行机制中自动执行，那么cucumber.json报告文件只记录这两个场景的结果，而不是全部5个场景。总体构建PASSES如果这两个场景在重新运行或FAILs中传递，如果这两个场景失败。这是正确的，但我的问题在于，通过重新运行机制覆盖了cucumber.json。
我曾尝试使用 maven-cucumber-reporting 插件v_3.16.0，但它实际上读取了cucumber.json文件，因此无法解决我的问题。任何帮助，将不胜感激。
I have 5 scenarios in my feature file. If all of them pass in 1st run. It successfully generates the report with cucumber.json report showing all 5 scenarios.
But, if (say) 2 out of 5 scenarios fails, and those gets executed automatically in a rerun mechanism, the cucumber.json report file only recording results of those two scenarios and not the all 5 scenarios. Overall build PASSES if those 2 scenarios passes in rerun or FAILs if those 2 scenarios fails. That's correct, but my problem is with the cucumber.json getting overwritten by rerun mechanism.
I have tried to use the maven-cucumber-reporting plugin v_3.16.0 but it actually reads the cucumber.json file itself and hence don't resolve my problem. Any help would be appreciated.
推荐答案
在这种情况下， AllTestsRunner 会反复运行，直到成功为止到 rerunFailingTestsCount 参数中提到的计数，或者在上次运行时失败。因此，最后一次运行总是获胜。
In this case the AllTestsRunner is run repeatedly till it succeeds up to the count mentioned in the rerunFailingTestsCount parameter or fails on last run. So the last run always wins.
surefire插件在目标文件夹中创建自己的报告。两个报告是 AllTestsRunner.txt ，这是一个摘要， TEST-AllTestsRunner.xml 有详细信息。这两份报告都有关于所有运行的详细信息您可以创建一个自定义程序，将 TEST-AllTestsRunner.xml 文件转换为所需的json。
The surefire plugin creates its own reports in the target folder. Two reports are AllTestsRunner.txt, which is a summary, and TEST-AllTestsRunner.xml has the details. Both these reports has details about all runs. You could create a custom program to convert the TEST-AllTestsRunner.xml file to desired json.
有一个 surefire报告插件，它读取上述xml文件和生成HTML报告。它将在站点文件夹中创建报告并使用mvn站点运行。也许这样可行。
There is a surefire report plugin which reads the above xml file and generates html report. It will create reports in the site folder and run with mvn site. Maybe this works.
这篇关于cucumber.json报告被重新运行场景报告覆盖的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
