Add Pause/Resume Auto Follower APIs #47510

tlrx · 2019-10-03T13:47:25Z

This pull request adds two APIs that allow to pause and resume CCR auto-follower patterns:

// pause auto-follower
POST /_ccr/auto_follow/my_pattern/pause

// resume auto-follower
POST /_ccr/auto_follow/my_pattern/resume

The ability to pause and resume auto-follow patterns can be useful in some situations, including the rolling upgrades of cluster using a bi-directional cross-cluster replication scheme (see #46665).

This pull request adds a new active flag to the AutoFollowPattern and adapts the AutoCoordinator and AutoFollower classes so that it stops to fetch remote's cluster state when all auto-follow patterns associate to the remote cluster are paused.

When an auto-follower is paused, remote indices that match the pattern are just ignored: they are not added to the pattern's followed indices uids list that is maintained in the local cluster state. This way, when the auto-follow pattern is resumed the indices created in the remote cluster in the meantime will be picked up again and added as new following indices. Indices created and then deleted in the remote cluster will be ignored as they won't be seen at all by the auto-follower pattern at resume time.

As this pull request is already massive, it does not include documentation.

elasticmachine · 2019-10-03T13:47:27Z

Pinging @elastic/es-distributed (:Distributed/CCR)

tlrx · 2019-10-03T14:11:25Z

Marking as work in progress until tests are green.

tlrx · 2019-10-04T07:38:56Z

@elasticmachine update branch

tlrx · 2019-10-04T09:06:50Z

This is ready for review.

tlrx · 2019-10-08T06:38:57Z

@elasticmachine update branch

martijnvg

Took a quick look and left a 2 nits and a question. Looks great, so LGTM.

x-pack/plugin/ccr/src/main/java/org/elasticsearch/xpack/ccr/action/AutoFollowCoordinator.java

martijnvg · 2019-10-08T08:01:24Z

x-pack/plugin/ccr/src/main/java/org/elasticsearch/xpack/ccr/action/AutoFollowCoordinator.java

            final Thread thread = Thread.currentThread();
-            getRemoteClusterState(remoteCluster, metadataVersion + 1, (remoteClusterStateResponse, remoteError) -> {
+            getRemoteClusterState(remoteCluster, Math.max(1L, nextMetadataVersion), (remoteClusterStateResponse, remoteError) -> {


why is Math.max(1L, nextMetadataVersion) needed?

Now I'm reading it again, I'm not sure it is really needed. I'll dig into that.

jasontedor

Thanks @tlrx, the production code looks great. I left one comment about a method override.

jasontedor · 2019-10-08T10:10:54Z

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ccr/AutoFollowMetadata.java

+        }
+
+        // for testing purpose
+        public AutoFollowPattern(String remoteCluster,


It looks like this was only added for avoiding changing call sites but I'd rather we didn't. You can use IntelliJ refactoring (change signature on this method to add the boolean active parameter as the fourth argument setting the default value to true, and ignore the conflict with the other override) to automatically handle the call sites (although you might have to worry about line-wrapping). Then we can avoid adding this method.

Thanks Jason for spotting this, it should not have been committed. I added this ctor to avoid some work and planned to come back to it later, but didn't 🤦‍♂️

I pushed 0a551a2

tlrx · 2019-10-10T15:03:04Z

@elasticmachine run elasticsearch-ci/2
@elasticmachine run elasticsearch-ci/packaging-sample

dnhatn

LGTM. Thanks @tlrx.

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ccr/AutoFollowStats.java

dnhatn · 2019-10-10T17:35:20Z

run elasticsearch-ci/packaging-sample

tlrx · 2019-10-11T10:34:08Z

Thanks a lot @martijnvg, @jasontedor and @dnhatn

cjcenizal · 2019-10-11T17:26:48Z

@tlrx Great work on this! I've created a Kibana issue to update the UI to surface this functionality (elastic/kibana#48008). Is the active state returned as part of the GET response? If not, is there another way to retrieve it? We will need this state on the UI so that we will know which button (pause or resume) we should show the user.

This commit adds two APIs that allow to pause and resume CCR auto-follower patterns: // pause auto-follower POST /_ccr/auto_follow/my_pattern/pause // resume auto-follower POST /_ccr/auto_follow/my_pattern/resume The ability to pause and resume auto-follow patterns can be useful in some situations, including the rolling upgrades of cluster using a bi-directional cross-cluster replication scheme (see elastic#46665). This committ adds a new active flag to the AutoFollowPattern and adapts the AutoCoordinator and AutoFollower classes so that it stops to fetch remote's cluster state when all auto-follow patterns associate to the remote cluster are paused. When an auto-follower is paused, remote indices that match the pattern are just ignored: they are not added to the pattern's followed indices uids list that is maintained in the local cluster state. This way, when the auto-follow pattern is resumed the indices created in the remote cluster in the meantime will be picked up again and added as new following indices. Indices created and then deleted in the remote cluster will be ignored as they won't be seen at all by the auto-follower pattern at resume time. Backport of elastic#47510 for 7.x

This commit adds two APIs that allow to pause and resume CCR auto-follower patterns: // pause auto-follower POST /_ccr/auto_follow/my_pattern/pause // resume auto-follower POST /_ccr/auto_follow/my_pattern/resume The ability to pause and resume auto-follow patterns can be useful in some situations, including the rolling upgrades of cluster using a bi-directional cross-cluster replication scheme (see #46665). This commit adds a new active flag to the AutoFollowPattern and adapts the AutoCoordinator and AutoFollower classes so that it stops to fetch remote's cluster state when all auto-follow patterns associate to the remote cluster are paused. When an auto-follower is paused, remote indices that match the pattern are just ignored: they are not added to the pattern's followed indices uids list that is maintained in the local cluster state. This way, when the auto-follow pattern is resumed the indices created in the remote cluster in the meantime will be picked up again and added as new following indices. Indices created and then deleted in the remote cluster will be ignored as they won't be seen at all by the auto-follower pattern at resume time. Backport of #47510 for 7.x

tlrx · 2019-10-13T09:07:36Z

@cjcenizal as said via another channel, the state is exposed in the Get Auto-Follower API under the active boolean field.

This commit adds two APIs that allow to pause and resume CCR auto-follower patterns: // pause auto-follower POST /_ccr/auto_follow/my_pattern/pause // resume auto-follower POST /_ccr/auto_follow/my_pattern/resume The ability to pause and resume auto-follow patterns can be useful in some situations, including the rolling upgrades of cluster using a bi-directional cross-cluster replication scheme (see elastic#46665). This committ adds a new active flag to the AutoFollowPattern and adapts the AutoCoordinator and AutoFollower classes so that it stops to fetch remote's cluster state when all auto-follow patterns associate to the remote cluster are paused. When an auto-follower is paused, remote indices that match the pattern are just ignored: they are not added to the pattern's followed indices uids list that is maintained in the local cluster state. This way, when the auto-follow pattern is resumed the indices created in the remote cluster in the meantime will be picked up again and added as new following indices. Indices created and then deleted in the remote cluster will be ignored as they won't be seen at all by the auto-follower pattern at resume time.

This commit adds support for Pause/Resume Auto-Follower APIs to the HLRC, with the documentation. Relates #47510

Relates #47510 Co-Authored-By: James Rodewig <[email protected]>

Add Pause/Resume Auto Follower APIs

2e59148

tlrx added >enhancement :Distributed Indexing/CCR Issues around the Cross Cluster State Replication features v8.0.0 labels Oct 3, 2019

tlrx mentioned this pull request Oct 3, 2019

API to pause auto-follow patterns #46665

Closed

tlrx added the WIP label Oct 3, 2019

Skip Rest tests

c1bdf88

jasontedor added the release highlight label Oct 3, 2019

more fixes

fd72431

Merge branch 'master' into pause-auto-followers

98c63ad

tlrx removed the WIP label Oct 4, 2019

tlrx requested review from martijnvg, dnhatn and jasontedor October 4, 2019 09:06

Merge branch 'master' into pause-auto-followers

0348221

martijnvg approved these changes Oct 8, 2019

View reviewed changes

jasontedor approved these changes Oct 8, 2019

View reviewed changes

tlrx added 3 commits October 9, 2019 14:38

mvg feedback

e79c8f3

jason feedback

0a551a2

Merge branch 'master' into pause-auto-followers

c985832

dnhatn approved these changes Oct 10, 2019

View reviewed changes

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ccr/AutoFollowStats.java Show resolved Hide resolved

ywelsch added the v7.5.0 label Oct 11, 2019

tlrx merged commit 8b82e62 into elastic:master Oct 11, 2019

tlrx deleted the pause-auto-followers branch October 11, 2019 10:33

tlrx added the backport pending label Oct 11, 2019

tlrx mentioned this pull request Oct 11, 2019

Add Pause/Resume Auto Follower APIs #47904

Merged

cjcenizal mentioned this pull request Oct 11, 2019

Pause/resume auto-follow patterns elastic/kibana#48008

Closed

tlrx removed the backport pending label Oct 13, 2019

This was referenced Oct 14, 2019

Add documentation for Pause/Resume Auto-Follower APIs #47985

Merged

Add Pause/Resume Auto-Follower APIs to High Level REST Client #47989

Merged

tlrx added a commit that referenced this pull request Oct 14, 2019

Add Pause/Resume Auto-Follower APIs to High Level REST Client (#47989)

2891783

This commit adds support for Pause/Resume Auto-Follower APIs to the HLRC, with the documentation. Relates #47510

tlrx added a commit that referenced this pull request Oct 14, 2019

Add Pause/Resume Auto-Follower APIs to High Level REST Client (#48004)

e4ea8b4

This commit adds support for Pause/Resume Auto-Follower APIs to the HLRC, with the documentation. Relates #47510

tlrx added a commit that referenced this pull request Oct 15, 2019

Add documentation for Pause/Resume Auto-Follower APIs (#47985)

58ca9ee

Relates #47510 Co-Authored-By: James Rodewig <[email protected]>

tlrx mentioned this pull request Oct 15, 2019

Add documentation for Pause/Resume Auto-Follower APIs (#47985) #48034

Merged

tlrx added a commit that referenced this pull request Oct 15, 2019

Add documentation for Pause/Resume Auto-Follower APIs (#47985) (#48034)

fe15d9f

Relates #47510 Co-Authored-By: James Rodewig <[email protected]>

Mpdreamz mentioned this pull request Nov 19, 2019

[meta] 7.5 release elastic/elasticsearch-net#4232

Closed

24 tasks

codebrain mentioned this pull request Dec 16, 2019

Add Pause/Resume Auto Follower APIs elastic/elasticsearch-net#4273

Merged

jakelandis added v8.0.0-alpha1 and removed v8.0.0 labels Jul 26, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Pause/Resume Auto Follower APIs #47510

Add Pause/Resume Auto Follower APIs #47510

tlrx commented Oct 3, 2019 •

edited

Loading

elasticmachine commented Oct 3, 2019

tlrx commented Oct 3, 2019

tlrx commented Oct 4, 2019

tlrx commented Oct 4, 2019

tlrx commented Oct 8, 2019

martijnvg left a comment

martijnvg Oct 8, 2019

tlrx Oct 9, 2019

jasontedor left a comment

jasontedor Oct 8, 2019

tlrx Oct 9, 2019 •

edited

Loading

tlrx commented Oct 10, 2019

dnhatn left a comment

dnhatn commented Oct 10, 2019

tlrx commented Oct 11, 2019

cjcenizal commented Oct 11, 2019

tlrx commented Oct 13, 2019

Add Pause/Resume Auto Follower APIs #47510

Add Pause/Resume Auto Follower APIs #47510

Conversation

tlrx commented Oct 3, 2019 • edited Loading

elasticmachine commented Oct 3, 2019

tlrx commented Oct 3, 2019

tlrx commented Oct 4, 2019

tlrx commented Oct 4, 2019

tlrx commented Oct 8, 2019

martijnvg left a comment

Choose a reason for hiding this comment

martijnvg Oct 8, 2019

Choose a reason for hiding this comment

tlrx Oct 9, 2019

Choose a reason for hiding this comment

jasontedor left a comment

Choose a reason for hiding this comment

jasontedor Oct 8, 2019

Choose a reason for hiding this comment

tlrx Oct 9, 2019 • edited Loading

Choose a reason for hiding this comment

tlrx commented Oct 10, 2019

dnhatn left a comment

Choose a reason for hiding this comment

dnhatn commented Oct 10, 2019

tlrx commented Oct 11, 2019

cjcenizal commented Oct 11, 2019

tlrx commented Oct 13, 2019

tlrx commented Oct 3, 2019 •

edited

Loading

tlrx Oct 9, 2019 •

edited

Loading