Skip to content

[CI] Intermittent timeout in xpack ReadActionsTests #53340

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
williamrandolph opened this issue Mar 10, 2020 · 5 comments
Closed

[CI] Intermittent timeout in xpack ReadActionsTests #53340

williamrandolph opened this issue Mar 10, 2020 · 5 comments
Assignees
Labels
:Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC Team:Security Meta label for security team >test-failure Triaged test failures from CI

Comments

@williamrandolph
Copy link
Contributor

I've noticed an occasional timeout in ReadActionTests. It doesn't reproduce for me locally, but it's popped up a few times recently:

10 March:

REPRODUCE WITH: ./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.security.authz.ReadActionsTests.testWildcardExclusion" \
  -Dtests.seed=254EEC6E8CC43E34 \
  -Dtests.security.manager=true \
  -Dtests.locale=mk-MK \
  -Dtests.timezone=America/Argentina/Cordoba \
  -Dcompiler.java=13

9 March:

8 March:

In each case the logs are extensive and somewhat different, but they all result in thread leaks and a suite timeout.

@williamrandolph williamrandolph added >test-failure Triaged test failures from CI :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC labels Mar 10, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-security (:Security/Authorization)

@henningandersen
Copy link
Contributor

henningandersen commented Mar 19, 2020

Two more failures here:

https://gradle-enterprise.elastic.co/s/4l3ynd36fhgro/tests/failed
https://gradle-enterprise.elastic.co/s/yjst47s2pqexg/tests/failed#:x-pack:plugin:security-test-org.elasticsearch.xpack.security.authz-readactionstests

Unable to reproduce using:

./gradlew ':x-pack:plugin:security:test' --tests "org.elasticsearch.xpack.security.authz.ReadActionsTests"   -Dtests.seed=7EA552089271B256   -Dtests.security.manager=true   -Dtests.locale=ar-DZ   -Dtests.timezone=Pacific/Pohnpei   -Dcompiler.java=13

but it did take 3 minutes on my local CI, and I wonder if this could turn into 20 minutes on jenkins under load?

tlrx added a commit to tlrx/elasticsearch that referenced this issue Mar 25, 2020
tlrx added a commit to tlrx/elasticsearch that referenced this issue Mar 25, 2020
@tlrx
Copy link
Member

tlrx commented Mar 25, 2020

These tests timeout regularly (see build stats) and again today on master.

Following test triage rules, I'm going to mute the tests on master, 7.x and 7.6 and assign this issue.

tlrx added a commit that referenced this issue Mar 25, 2020
tlrx added a commit that referenced this issue Mar 25, 2020
tlrx added a commit that referenced this issue Mar 25, 2020
@rjernst rjernst added the Team:Security Meta label for security team label May 4, 2020
tvernum added a commit to tvernum/elasticsearch that referenced this issue Jul 12, 2021
Enable this integration test again, with a set timeout on each search
action.

Relates: elastic#53340
tvernum added a commit that referenced this issue Jul 12, 2021
Enable this integration test again, with a set timeout on each search
action.

Relates: #53340
@tvernum
Copy link
Contributor

tvernum commented Jul 13, 2021

I've enabled this test again with some better timeouts.
If it fails again, please mute aggressively and ping on this issue.

tvernum added a commit to tvernum/elasticsearch that referenced this issue Sep 22, 2021
Enable this integration test again, with a set timeout on each search
action.

Relates: elastic#53340
Backport of: elastic#75208
elasticsearchmachine pushed a commit that referenced this issue Sep 22, 2021
Enable this integration test again, with a set timeout on each search
action.

Relates: #53340
Backport of: #75208
@gwbrown
Copy link
Contributor

gwbrown commented Oct 28, 2022

This test has been re-enabled for over a year, no new failures have been reported here, and I don't see any failures in Gradle Enterprise. I'm going to go ahead and close this.

@gwbrown gwbrown closed this as completed Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC Team:Security Meta label for security team >test-failure Triaged test failures from CI
Projects
None yet
Development

No branches or pull requests

7 participants