Test Flake: A Jasmine spec timed out. Resetting the WebDriver Control Flow.

[spadgett]

Running "protractor:default" (protractor) task
Starting selenium standalone server...
[launcher] Running 1 instances of WebDriver
Selenium standalone server started at http://172.18.14.245:57776/wd/hub
Spec started
visiting url: create-project
visiting url: project/console-test-project-1506967814825/create
visiting url: project/console-test-project-1506967814825/browse/images
A Jasmine spec timed out. Resetting the WebDriver Control Flow.
The last active task was: 
Protractor.get(https://localhost:9000/dev-console/project/console-test-project-1506967814825/browse/images) - reset url

[spadgett]

https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_origin_web_console/305

[spadgett]

Screenshot is just a loading screen:

cc @benjaminapetersen

[benjaminapetersen]

Yeah I think in all this e2e stuff the dependencies aren't nailed down as well as we would like. Selenium or webdriver versions seems to suddenly change and things get funny. :/

[spadgett]

Selenium or webdriver versions seems to suddenly change and things get funny.

We probably should commit package-lock.json to avoid this stuff.

[benjaminapetersen]

Re: #2083

[spadgett]

Might be unrelated, but Firefox recently changed from

52.3.0-2.el7_4 -> 52.4.0-1.el7_4

[spadgett]

https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_origin_web_console/304/
https://ci.openshift.redhat.com/jenkins/job/test_pull_request_origin_web_console/244/

It's failing on Protractor.get, but not every time. I wonder if we just need to bump the timeout.

[spadgett]

Timeout is only 3s. I can see the page taking longer than that to load. Since the tests take 20-30 minutes to run in Jenkins, we might want to bump this to something much higher.

Error: Wait timed out after 3050ms

[benjaminapetersen]

I think its in here:

Spec started
visiting url: create-project
visiting url: project/console-test-project-1506965913119/create
visiting url: project/console-test-project-1506965913119/browse/images
A Jasmine spec timed out. Resetting the WebDriver Control Flow.
The last active task was: 
Protractor.get(https://localhost:9000/dev-console/project/console-test-project-1506965913119/browse/images) - reset url

Tinkering locally, for some reason the click on "create" is no longer happening before the browser navigates to the next page (images). So it never actually creates the resources, thus the tables are empty.

Thats the issue I've been working on the past 2 days locally, now its showing up in Travis.

[benjaminapetersen]

Here: https://github.com/openshift/origin-web-console/pull/2083/files#diff-74ba08d7a3f605db52c0916bda85d917R23

[spadgett]

getPageTimeout is set to 30s, so why is it timing out after 3s?

https://github.com/openshift/origin-web-console/blob/master/test/protractor.conf.js#L180

[benjaminapetersen]

I think its not waiting for anything, its just plowing through the rest of the tests and failing. The A Jasmine spec timed out. Resetting the WebDriver Control Flow. doesn't quite seem to be truthful...

[spadgett]

Well we have waitForUri, but that's supposed to wait 5s. So where is 3s coming from?

origin-web-console/test/integration/helpers.js

Lines 102 to 106 in 9c144f3

	exports.goToPage = function(uri) {
	return browser.get(uri).then(() => {
	return waitForUri(uri);
	});
	};

origin-web-console/test/integration/helpers.js

Lines 50 to 59 in 9c144f3

	var waitForUri = function(expectedUri) {
	let actualUrl;
	return browser.wait(() => {
	return browser.getCurrentUrl().then((url) => {
	actualUrl = url;
	return actualUrl.indexOf(expectedUri) > -1;
	});
	}, 5000, "URL hasn't changed to " + expectedUri + '(is currently: ' + actualUrl + ')');
	};
	exports.waitForUri = waitForUri;

[spadgett]

Testing an increase in the waitForUri timeout: #2203

[benjaminapetersen]

Nice, tinkering as well.

[benjaminapetersen]

My theory, I think its deeper. I don't believe it is timing out in the navigation, I think its timing out waiting for elements to be rendered to the page. Since protractor is in sync w/Angular, it knowns when Angular is done (all promises resolved) for requesting the objects that never get created. So it is rendering empty tables, then the element(by.cssContainingText()) never resolve to anything.

However, your links above consistently timeout at 3050, so thats weird.

[spadgett]

@benjaminapetersen I don't think you're wrong, but I think you're describing a different flake (#1685). This particular error consistently shows the white "Loading..." page in failure screenshots like discovery hasn't finished. I think it's just slow and timing out too soon.

I wonder if our pre-Angular bootstrap tasks are tripping Protractor up.

[spadgett]

https://github.com/angular/protractor/blob/1.7.0/docs/timeouts.md

[benjaminapetersen]

Oh that might be. I've read about custom bootstrapping causing problems. Will see if I can dig up the reference.

[spadgett]

I'm pretty sure this was caused by the Firefox 52.3.0 -> 52.4.0 bump. Both the test and merge queues stopped working as soon as the installed Firefox package was updated. 52.4.0 is the latest ESR released a few days ago.

[spadgett]

Opened openshift-eng/aos-cd-jobs#704 to temporarily downgrade Firefox as a workaround

[benjaminapetersen]

Jenkins w/Firefox:

[spadgett]

Downgrading FF did not fix the tests :(

[spadgett]

Lowering severity as the tests are no longer flaking.

[benjaminapetersen]

Updating timeouts in PR #2083 here https://github.com/openshift/origin-web-console/pull/2083/files#diff-1a86c750617c5201796c35f6ba135f91R39
and here https://github.com/openshift/origin-web-console/pull/2083/files#diff-53424273570825c8ca0520f259622910R1

[spadgett]

This failure is no longer happening.

[ambicamahapatro]

Getting the error Jasmine spec timed out. Resetting the WebDriver Control Flow while executing the test suit but individually it runs fine. tried changing the protractor config defaultTimeoutInterval and changing the Jasmine framework to Jasmine from jasmine2. Still nothing works. can somebody plz help me