Rerun Functionality

[sugatmankar]

Hello Temyers,

It is my great pleasure to contribute rerun functionality.

Please have a look at it. let me know if i miss something.

Thanks & Regards
Sugad Mankar

[temyers]

Thanks @sugatmankar.

I've had a look. I'll add comments directly.

There are currently merge conflicts - please can you merge out and resolve them.

[temyers]

Ok, there are now two properties defining which template to use: useTestNG and useReRun.

This adds complication - should rerun + testNG be supported together?

[sugatmankar]

I didn't try for TestNG. Once i'm done with current changes. Will look for TestNG as well. :)

[Balhindi]

I run into network issues all the time, it is not that they are flaky tests, but rather flaky environments or sometimes bad luck with the configuration.

[saikrishna321]

does this support only junit runner ?

[sugatmankar]

@saikrishna321 for now we are supporting only for junit.

[saikrishna321]

@sugatmankar i think TestNG equally is important, would it possible to have it within this PR ?

[sugatmankar]

I will try for TestNG as well. Currently i'm focusing on junit re-runner with tag wise parallel. Because current implementation is like if we selected 10 tags from feature1.feature and 10 tags from feature2.feature, total 20 tags are there but only 2 parallel thread will be executed. Correct me if m wrong. so i have implemented IT generator which will generate IT file as per number of tags, it will also search for feature file where tag is present and only this path will be added in arg as "classpath:features/test1.feature". with this implementation i'm able to use rerun with tag wise parallel execution.

[JasonSmiley208]

I was going to make a patch for this pull request, but I do not have push access.

[sugatmankar]

@saikrishna i designed reruner by direct invoking cli class of cucumber. So
it is not specific to junit or testng. It is generic. TestNG use Abstract
Test ng listner which internally calles cli and junits
@RunWith(cucumber.class) calles same cli class implementation.

So its common for both. Instead JunitRerun prop i will rename this as
cliReRun.

@JSON please fork this PR. This will create repo in your GitHub then play
with it, you will have all access in your acc.

Apologize guys. :( I am out of town for few days. Now i can only read and
reply to mails. Once m back will fix all asap.

On Fri 27 May, 2016, 7:42 PM Jason Smiley, [email protected] wrote:

I was going to make a patch for this pull request, but I do not have push
access.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#30 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AEFbnc7DqM3oCd7mWLarEQGwsATSihRGks5qFvu6gaJpZM4IY7YE
.

[JasonSmiley208]

Instead JunitRerun prop i will rename this as
cliReRun.

Or just rerunCount?

[sugatmankar]

Please suggest which one will be appropriate?

On Fri 27 May, 2016, 8:31 PM Jason Smiley, [email protected] wrote:

"Instead JunitRerun prop i will rename this as
cliReRun."

Or just rerunCount?

On Fri, May 27, 2016 at 10:36 AM, sugat mankar [email protected]
wrote:

@saikrishna i designed reruner by direct invoking cli class of cucumber.
So
it is not specific to junit or testng. It is generic. TestNG use Abstract
Test ng listner which internally calles cli and junits
@RunWith(cucumber.class) calles same cli class implementation.

So its common for both. Instead JunitRerun prop i will rename this as
cliReRun.

@JSON please fork this PR. This will create repo in your GitHub then play
with it, you will have all access in your acc.

Apologize guys. :( I am out of town for few days. Now i can only read and
reply to mails. Once m back will fix all asap.

On Fri 27 May, 2016, 7:42 PM Jason Smiley, [email protected]
wrote:

I was going to make a patch for this pull request, but I do not have
push
access.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
<

#30 (comment)

,
or mute the thread
<

https://github.com/notifications/unsubscribe/AEFbnc7DqM3oCd7mWLarEQGwsATSihRGks5qFvu6gaJpZM4IY7YE

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
#30 (comment)
,
or mute the thread
<
https://github.com/notifications/unsubscribe/AP8wZX6zxwrMTPa6PoaPhl9YdudRM5Dbks5qFwF_gaJpZM4IY7YE

.

Jason Smiley
Software Developer in Test

90 Fifth Avenue 3rd Floor
New York, NY 10011
[image: Compass] http://www.compass.com

NEW YORK | BROOKLYN | EAST HAMPTON | BRIDGEHAMPTON | SOUTHAMPTON | SAG
HARBOR | WASHINGTON DC | CHEVY CHASE | BOSTON | CAMBRIDGE | MIAMI | COCONUT
GROVE | BEVERLY HILLS | MALIBU | PASADENA | MONTECITO | SANTA BARBARA |
BASALT | ASPEN

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#30 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AEFbnVcHaOGwfKPBfMpWBZHHf9kTg9Vyks5qFwdHgaJpZM4IY7YE
.

[JasonSmiley208]

I believe that if this property is being used to determine how times we should rerun failing tests (regardless of which runner we are using), then I say we call it "rerunCount" so its generic and people want to use it.

If its tied to a runner, then "RerunCount". Correct me if I am wrong, this only supports Junit and TestNG and nothing else (both of which use this property), so I say call it "rerunCount".

[sugatmankar]

Yeah i need to restore it as "ReRunCount" and need to remove @test tag in
rerun velocity template and create main method psvm. Which will remove
confusion about junit/ testng .

On Fri 27 May, 2016, 9:41 PM Jason Smiley, [email protected] wrote:

I believe that if this property is being used to determine how times we
should rerun failing tests (regardless of which runner we are using), then
I say we call it "rerunCount".

If its tied to a runner, then "RerunCount". Correct me if I am wrong, this
only supports Junit and TestNG and nothing else (both of which use this
property), so I say call it "rerunCount".

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#30 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AEFbne83CAm0BhvN0fZXEdfWNbnn41NSks5qFxergaJpZM4IY7YE
.

[temyers]

@sugatmankar What's the status with this PR?

Need any help?

[sugatmankar]

@temyers added this functionality in tagwiseOutlinewise. You can link this with PR 53

[sugatmankar]

Fixed issue of rerun tagwiseOutlinewiseIssueRerun

To achieve Scenario Outline parallelism have to removed classpath: from velocity template.
if i kept feature path with classpath:features/feature1.feature:40 it wont run.
but if i run with src/test/resources/features/feature1.feature:40, wich will run and rerun without any problem. For that have to change featuresDirectory property from file to String and update changes wherever required.

[JasonSmiley208]

I see that we are up to 2.0.0 on the change log - was rerun functionality added to the project?

or am I missing something

[temyers]

Rerun hasn't been merged yet. The travis build is failing.

2.0.0 is still in SNAPSHOT.

[sugatmankar]

@JasonSmiley208 Please try Steps mentioned in comment here

[SasidharParnandi]

"cucumber-jvm-parallel-plugin" works just perfectly for me reducing our smoke tests run time on Jenkins to a great extent. I am happy with feature level parallel execution. Just the only limitation I find is missing support for "rerun". We do one time re-run of failed scenarios for few good reasons.

@sugatmankar : Does this rerun functionality create scenariowise runners for failed ones and keep original runners still be at feature level? As I am happy to have feature level parallel execution.

[sugatmankar]

@SasidharParnandi First of all main goal was to parallalize scenarios as well as scenario outlines also. Yes rerun functionality will available in all three parallalization ( Feature wise, Scenario wise , Scenario Outline wise ). Fork my repo as per steps mentioned above comments else you can wait for release 2.0.0

@temyers Do you need any help merging this functionality ?

[temyers]

@sugatmankar I'm in the process of reviewing and re-applying this.

Progress is slow as I haven't got much time to spend on this at the moment. I'll tidy up what I have and create a new pull request to compare.

[sugatmankar]

reimplemented in #89 .

[alinpa]

@sugatmankar Did you implement the same for TestNG runner? I am trying to run this but I cannot see the failed test to be run according to the rerunFailingTestsCount param.