Make writing of 'dummy' JUnit results configurable

[kutzi]

Summary

#567 introduced creation of 'dummy' JUnit results if no feature matches the passed filters (i.e. all are skipped). Would be good if that behaviour could be deactivated.

Expected Behavior

Apparently this was explicitely introduced as a workaround for issues with Jenkins (job configs)
See 0fcca0a
IMO it's not the responsiblity of cucumber-jvm to add workarounds for downstream tool 'issues' (actually it's not a Jenkins issue as well IMO, but a wrong job configuration)

IMHO, this would ideally not have been introduced at all, but since it's already a feature for so long, it's probably a bad idea to change the default behaviour back.
However, a (commandline) switch to turn this behaviour of would be good.

Also it would be better to have at least some more info what these 'dummy' results are about. Took me some time to track down from where they originated in my project.

[kutzi]

Having thought about this once more:
Please just remove this behaviour. Emitting some test results when actually no tests have run is just wrong.
Also this weird behaviour is documented nowhere.

IF someone comes up with a valid use case for that feature, then you could think about making it optional.

[mpkorstanje]

IMO it's not the responsiblity of cucumber-jvm to add workarounds for downstream tool 'issues' (actually it's not a Jenkins issue as well IMO, but a wrong job configuration)

It does indeed appear to be a configuration setting in the JUnit Plugin.

https://wiki.jenkins.io/display/JENKINS/JUnit+Plugin

Allow empty results: If checked, the default behavior of failing a build on missing test result files or empty test results is changed to not affect the status of the build. Please note that this setting make it harder to spot misconfigured jobs or build failures where the test tool does not exit with an error code when not producing test report files.

@brasmusson can you remember why it was added?

[brasmusson]

#567 was merged into Cucumber-JVM Aug 4, 2013. The option for allowing empty results was merged into the Jenkins JUnit Plugin Nov 24, 2015. So at the time the 'dummy' JUnit results was introduced into Cucumber-JVM it was solving a real problem.

[mpkorstanje]

@kutzi if you want to make a PR to remove it go ahead.

[kutzi]

Is there a chance that this can be backported to 2.0.x as well?
As AFAIK with cucumber-groovy I'm currently stuck with 2.0.x
Or can you explain to me how the current state of cucumber-groovy looks like?

[mlvandijk]

@kutzi The Groovy repo is looking for a new maintainer AFAIK. Would you be interested?

[kutzi]

@mlvandijk if someone has a rough overview about what to do to bring Cucumber Groovy to work with cucumber-jvm 2.3.x, I could try to help out.
But becoming a maintainer: sorry, no

[mlvandijk]

@kudzi You could check the CHANGELOG for a high overview of what was changed. And the commits for details. (Or the commits in your favorite IDE/tooling of choice ;)

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.