Skip to content

[CI] ConcurrentSeqNoVersioningIT testSeqNoCASLinearizability failing #88120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
luigidellaquila opened this issue Jun 28, 2022 · 2 comments · Fixed by #89926
Closed

[CI] ConcurrentSeqNoVersioningIT testSeqNoCASLinearizability failing #88120

luigidellaquila opened this issue Jun 28, 2022 · 2 comments · Fixed by #89926
Assignees
@fcofdez
Labels
:Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI

Comments

@luigidellaquila
Copy link
Contributor

Build scan:
https://gradle-enterprise.elastic.co/s/hdrkqs5hmpp6k/tests/:server:internalClusterTest/org.elasticsearch.versioning.ConcurrentSeqNoVersioningIT/testSeqNoCASLinearizability

Reproduction line:
./gradlew ':server:internalClusterTest' --tests "org.elasticsearch.versioning.ConcurrentSeqNoVersioningIT.testSeqNoCASLinearizability" -Dtests.seed=FF0D42993DBCEB07 -Dtests.locale=es-HN -Dtests.timezone=Chile/Continental -Druntime.java=18

Applicable branches:
8.3

Reproduces locally?:
No

Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.versioning.ConcurrentSeqNoVersioningIT&tests.test=testSeqNoCASLinearizability

Failure excerpt:

com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4099, name=elasticsearch[node_t3][generic][T#3], state=RUNNABLE, group=TGRP-ConcurrentSeqNoVersioningIT]


  Caused by: java.lang.AssertionError: supposedly in-sync shard copy received a global checkpoint [566] that is higher than its local checkpoint [560]

    at __randomizedtesting.SeedInfo.seed([FF0D42993DBCEB07]:0)
    at org.elasticsearch.index.shard.IndexShard.updateGlobalCheckpointOnReplica(IndexShard.java:2760)
    at org.elasticsearch.index.shard.IndexShard.lambda$innerAcquireReplicaOperationPermit$30(IndexShard.java:3537)
    at org.elasticsearch.index.shard.IndexShard$5.onResponse(IndexShard.java:3398)
    at org.elasticsearch.index.shard.IndexShard$5.onResponse(IndexShard.java:3368)
    at org.elasticsearch.index.shard.IndexShard.lambda$asyncBlockOperations$24(IndexShard.java:3319)
    at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:162)
    at org.elasticsearch.index.shard.IndexShardOperationPermits$1.doRun(IndexShardOperationPermits.java:107)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:769)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.lang.Thread.run(Thread.java:833)
@luigidellaquila luigidellaquila added :Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. >test-failure Triaged test failures from CI labels Jun 28, 2022
@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Jun 28, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@fcofdez fcofdez self-assigned this Jun 28, 2022
@ywangd
Copy link
Member

ywangd commented Jul 7, 2022

Failed again on main https://gradle-enterprise.elastic.co/s/avd2mnfc5dmys/tests/:server:internalClusterTest/org.elasticsearch.versioning.ConcurrentSeqNoVersioningIT/testSeqNoCASLinearizability?top-execution=1

fcofdez added a commit to fcofdez/elasticsearch that referenced this issue Sep 8, 2022
@fcofdez
Do not deserialize back BytesTransportRequest to clone a request in M…
c358446 
…ockTransportService

Some handlers, such as JOIN_VALIDATE_ACTION_NAME, deserialize a
BytesTransportRequest into a different class that cannot be serialized.
This commit changes that behaviour and directly clones the BytesTransportRequest
instead of relying on the transport handler.

Closes elastic#88120
@fcofdez fcofdez mentioned this issue Sep 8, 2022
Merged
@fcofdez fcofdez closed this as completed in 14d93e0 Sep 8, 2022
fcofdez added a commit to fcofdez/elasticsearch that referenced this issue Sep 8, 2022
@fcofdez
Do not deserialize back BytesTransportRequest to clone a request in M…
c9d371a 
…ockTransportService (elastic#89926)

Some handlers, such as JOIN_VALIDATE_ACTION_NAME, deserialize a
BytesTransportRequest into a different class that cannot be serialized.
This commit changes that behaviour and directly clones the BytesTransportRequest
instead of relying on the transport handler.

Closes elastic#88120
fcofdez added a commit that referenced this issue Sep 8, 2022
@fcofdez
Do not deserialize back BytesTransportRequest to clone a request in M…
ba317b4 
…ockTransportService (#89926) (#89936)

Some handlers, such as JOIN_VALIDATE_ACTION_NAME, deserialize a
BytesTransportRequest into a different class that cannot be serialized.
This commit changes that behaviour and directly clones the BytesTransportRequest
instead of relying on the transport handler.

Closes #88120
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fcofdez fcofdez

Labels
:Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
4 participants
@luigidellaquila @fcofdez @ywangd @elasticmachine