-
Notifications
You must be signed in to change notification settings - Fork 25.2k
Introduce transport API for cluster bootstrapping #34961
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
DaveCTurner
merged 58 commits into
elastic:zen2
from
DaveCTurner:2018-10-29-bootstrap-transport-api
Nov 8, 2018
Merged
Changes from 41 commits
Commits
Show all changes
58 commits
Select commit
Hold shift + click to select a range
b01d321
Introduce transport API for cluster bootstrapping
DaveCTurner c43f31f
Merge branch 'zen2' into 2018-10-29-bootstrap-transport-api
DaveCTurner 0cc2b20
Fix up post-merge
DaveCTurner fb5c0ff
Rename setters/getters
DaveCTurner 65e60b1
Javadocs
DaveCTurner 592c87e
Rename setter
DaveCTurner d73638b
Rename getter
DaveCTurner eb407f4
Rename timeout getter/setter
DaveCTurner 08dd103
Void setters
DaveCTurner 3c3b35a
Javadocs for GetDiscoveredNodesRequest
DaveCTurner 53e2a79
Javadocs for GetDiscoveredNodesResponse
DaveCTurner 47b3718
Move action to cluster:admin
DaveCTurner 5d97e23
Wrap clients for bootstraping in order to support security
DaveCTurner bdbd600
Rename waitForNodes -> minimumNodeCount
DaveCTurner 4a73b26
Writeable not Streamable
DaveCTurner 4a5083a
Force immutability
DaveCTurner 17e95d8
Remove blank line
DaveCTurner ca6d0b3
Writeable not Streamable
DaveCTurner 0f7f8fa
Not final
DaveCTurner 48e6096
Change lengths of ID/names
DaveCTurner 55994a0
Use verifyZeroInteractions
DaveCTurner ef9ae33
Remove draconian validation
DaveCTurner 8b776b3
Writeable not Streamable
DaveCTurner 7de67e2
Reword
DaveCTurner 8c3047c
Imports
DaveCTurner 45cd25e
Message
DaveCTurner 696b71f
Use Mockito
DaveCTurner f100de4
Message
DaveCTurner 1efbf70
Make use of default timeout
DaveCTurner d4818d6
Async impl using ListenableFuture
DaveCTurner 2fd41bb
Move setInitialConfiguration method to coordinator and call it on the…
DaveCTurner 4b50388
Don't throw if initial configuration already set
DaveCTurner 1ff3e0f
Tweak
DaveCTurner 1b62782
Extract and use scheduleUnlessShuttingDown
DaveCTurner 0a5a8a8
Register TransportBootstrapClusterAction with TransportService
DaveCTurner 3ee0174
Register GetDiscoveredNodesAction with TransportService
DaveCTurner 759ad07
Don't use AcknowledgedResponse
DaveCTurner fc2f125
Refactor test initialisation
DaveCTurner d0a62cb
Rename minimumNodeCount to waitForNodes
DaveCTurner ad31d90
Merge branch 'zen2' into 2018-10-29-bootstrap-transport-api
DaveCTurner abcecf6
Merge branch 'zen2' into 2018-10-29-bootstrap-transport-api
DaveCTurner 457cbc7
Use fields not getters
DaveCTurner 144c1a5
Specifically master-eligible nodes
DaveCTurner 00a2be3
Better exceptions
DaveCTurner e3637c5
AbstractRunnable
DaveCTurner d3766b4
Use direct executor
DaveCTurner 100ed18
Use ActionListener.wrap
DaveCTurner 2daf1b1
Implement getResponseReader
DaveCTurner 0f25ba2
Javadocs for setInitialConfiguration
DaveCTurner 2d411a5
Add TODO about adding local node to getFoundPeers
DaveCTurner ad63cee
Per-suite threadpool
DaveCTurner 8688b92
Parallelize bootstrap requests
DaveCTurner 276251e
Allow default of 30 seconds for startup
DaveCTurner 2325d4b
Imports
DaveCTurner 6d05c29
Better messages
DaveCTurner b957940
Merge branch 'zen2' into 2018-10-29-bootstrap-transport-api
DaveCTurner 7e2b550
TODO fix this
DaveCTurner eaba4a7
Compile fixes
DaveCTurner File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
35 changes: 35 additions & 0 deletions
35
...rc/main/java/org/elasticsearch/action/admin/cluster/bootstrap/BootstrapClusterAction.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
/* | ||
* Licensed to Elasticsearch under one or more contributor | ||
* license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright | ||
* ownership. Elasticsearch licenses this file to you under | ||
* the Apache License, Version 2.0 (the "License"); you may | ||
* not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, | ||
* software distributed under the License is distributed on an | ||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
* KIND, either express or implied. See the License for the | ||
* specific language governing permissions and limitations | ||
* under the License. | ||
*/ | ||
package org.elasticsearch.action.admin.cluster.bootstrap; | ||
|
||
import org.elasticsearch.action.Action; | ||
|
||
public class BootstrapClusterAction extends Action<BootstrapClusterResponse> { | ||
public static final BootstrapClusterAction INSTANCE = new BootstrapClusterAction(); | ||
public static final String NAME = "cluster:admin/bootstrap_cluster"; | ||
|
||
private BootstrapClusterAction() { | ||
super(NAME); | ||
} | ||
|
||
@Override | ||
public BootstrapClusterResponse newResponse() { | ||
throw new UnsupportedOperationException("usage of Streamable is to be replaced by Writeable"); | ||
} | ||
} |
65 changes: 65 additions & 0 deletions
65
...c/main/java/org/elasticsearch/action/admin/cluster/bootstrap/BootstrapClusterRequest.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,65 @@ | ||
/* | ||
* Licensed to Elasticsearch under one or more contributor | ||
* license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright | ||
* ownership. Elasticsearch licenses this file to you under | ||
* the Apache License, Version 2.0 (the "License"); you may | ||
* not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, | ||
* software distributed under the License is distributed on an | ||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
* KIND, either express or implied. See the License for the | ||
* specific language governing permissions and limitations | ||
* under the License. | ||
*/ | ||
package org.elasticsearch.action.admin.cluster.bootstrap; | ||
|
||
import org.elasticsearch.action.ActionRequest; | ||
import org.elasticsearch.action.ActionRequestValidationException; | ||
import org.elasticsearch.common.io.stream.StreamInput; | ||
import org.elasticsearch.common.io.stream.StreamOutput; | ||
|
||
import java.io.IOException; | ||
|
||
/** | ||
* Request to set the initial configuration of master-eligible nodes in a cluster so that the very first master election can take place. | ||
*/ | ||
public class BootstrapClusterRequest extends ActionRequest { | ||
private final BootstrapConfiguration bootstrapConfiguration; | ||
|
||
public BootstrapClusterRequest(BootstrapConfiguration bootstrapConfiguration) { | ||
this.bootstrapConfiguration = bootstrapConfiguration; | ||
} | ||
|
||
public BootstrapClusterRequest(StreamInput in) throws IOException { | ||
super(in); | ||
bootstrapConfiguration = new BootstrapConfiguration(in); | ||
} | ||
|
||
/** | ||
* @return the bootstrap configuration: the initial set of master-eligible nodes whose votes are counted in elections. | ||
*/ | ||
public BootstrapConfiguration getBootstrapConfiguration() { | ||
return bootstrapConfiguration; | ||
} | ||
|
||
@Override | ||
public ActionRequestValidationException validate() { | ||
return null; | ||
} | ||
|
||
@Override | ||
public void readFrom(StreamInput in) throws IOException { | ||
throw new UnsupportedOperationException("usage of Streamable is to be replaced by Writeable"); | ||
} | ||
|
||
@Override | ||
public void writeTo(StreamOutput out) throws IOException { | ||
super.writeTo(out); | ||
bootstrapConfiguration.writeTo(out); | ||
} | ||
} |
66 changes: 66 additions & 0 deletions
66
.../main/java/org/elasticsearch/action/admin/cluster/bootstrap/BootstrapClusterResponse.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
/* | ||
* Licensed to Elasticsearch under one or more contributor | ||
* license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright | ||
* ownership. Elasticsearch licenses this file to you under | ||
* the Apache License, Version 2.0 (the "License"); you may | ||
* not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, | ||
* software distributed under the License is distributed on an | ||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
* KIND, either express or implied. See the License for the | ||
* specific language governing permissions and limitations | ||
* under the License. | ||
*/ | ||
package org.elasticsearch.action.admin.cluster.bootstrap; | ||
|
||
import org.elasticsearch.action.ActionResponse; | ||
import org.elasticsearch.common.io.stream.StreamInput; | ||
import org.elasticsearch.common.io.stream.StreamOutput; | ||
|
||
import java.io.IOException; | ||
|
||
/** | ||
* Response to a {@link BootstrapClusterRequest} indicating that the cluster has been successfully bootstrapped. | ||
*/ | ||
public class BootstrapClusterResponse extends ActionResponse { | ||
private final boolean alreadyBootstrapped; | ||
|
||
public BootstrapClusterResponse(boolean alreadyBootstrapped) { | ||
this.alreadyBootstrapped = alreadyBootstrapped; | ||
} | ||
|
||
public BootstrapClusterResponse(StreamInput in) throws IOException { | ||
super(in); | ||
alreadyBootstrapped = in.readBoolean(); | ||
} | ||
|
||
/** | ||
* @return whether this node already knew that the cluster had been bootstrapped when handling this request. | ||
*/ | ||
public boolean getAlreadyBootstrapped() { | ||
return alreadyBootstrapped; | ||
} | ||
|
||
@Override | ||
public void readFrom(StreamInput in) throws IOException { | ||
throw new UnsupportedOperationException("usage of Streamable is to be replaced by Writeable"); | ||
} | ||
|
||
@Override | ||
public void writeTo(StreamOutput out) throws IOException { | ||
super.writeTo(out); | ||
out.writeBoolean(alreadyBootstrapped); | ||
} | ||
|
||
@Override | ||
public String toString() { | ||
return "BootstrapClusterResponse{" + | ||
"alreadyBootstrapped=" + alreadyBootstrapped + | ||
'}'; | ||
} | ||
} |
179 changes: 179 additions & 0 deletions
179
...rc/main/java/org/elasticsearch/action/admin/cluster/bootstrap/BootstrapConfiguration.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,179 @@ | ||
/* | ||
* Licensed to Elasticsearch under one or more contributor | ||
* license agreements. See the NOTICE file distributed with | ||
* this work for additional information regarding copyright | ||
* ownership. Elasticsearch licenses this file to you under | ||
* the Apache License, Version 2.0 (the "License"); you may | ||
* not use this file except in compliance with the License. | ||
* You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, | ||
* software distributed under the License is distributed on an | ||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
* KIND, either express or implied. See the License for the | ||
* specific language governing permissions and limitations | ||
* under the License. | ||
*/ | ||
package org.elasticsearch.action.admin.cluster.bootstrap; | ||
|
||
import org.elasticsearch.ElasticsearchException; | ||
import org.elasticsearch.cluster.ClusterState.VotingConfiguration; | ||
import org.elasticsearch.cluster.node.DiscoveryNode; | ||
import org.elasticsearch.common.Nullable; | ||
import org.elasticsearch.common.io.stream.StreamInput; | ||
import org.elasticsearch.common.io.stream.StreamOutput; | ||
import org.elasticsearch.common.io.stream.Writeable; | ||
|
||
import java.io.IOException; | ||
import java.util.ArrayList; | ||
import java.util.Collections; | ||
import java.util.HashSet; | ||
import java.util.List; | ||
import java.util.Objects; | ||
import java.util.Set; | ||
import java.util.stream.Collectors; | ||
|
||
public class BootstrapConfiguration implements Writeable { | ||
|
||
private final List<NodeDescription> nodeDescriptions; | ||
|
||
public BootstrapConfiguration(List<NodeDescription> nodeDescriptions) { | ||
if (nodeDescriptions.isEmpty()) { | ||
throw new IllegalArgumentException("cannot create empty bootstrap configuration"); | ||
} | ||
this.nodeDescriptions = Collections.unmodifiableList(new ArrayList<>(nodeDescriptions)); | ||
} | ||
|
||
public BootstrapConfiguration(StreamInput in) throws IOException { | ||
nodeDescriptions = Collections.unmodifiableList(in.readList(NodeDescription::new)); | ||
assert nodeDescriptions.isEmpty() == false; | ||
} | ||
|
||
public List<NodeDescription> getNodeDescriptions() { | ||
return nodeDescriptions; | ||
} | ||
|
||
public VotingConfiguration resolve(Iterable<DiscoveryNode> discoveredNodes) { | ||
final Set<DiscoveryNode> selectedNodes = new HashSet<>(); | ||
for (final NodeDescription nodeDescription : nodeDescriptions) { | ||
final DiscoveryNode discoveredNode = nodeDescription.resolve(discoveredNodes); | ||
if (selectedNodes.add(discoveredNode) == false) { | ||
throw new ElasticsearchException("multiple nodes matching {} in {}", discoveredNode, this); | ||
} | ||
} | ||
|
||
final Set<String> nodeIds = selectedNodes.stream().map(DiscoveryNode::getId).collect(Collectors.toSet()); | ||
assert nodeIds.size() == selectedNodes.size() : selectedNodes + " does not contain distinct IDs"; | ||
return new VotingConfiguration(nodeIds); | ||
} | ||
|
||
@Override | ||
public void writeTo(StreamOutput out) throws IOException { | ||
out.writeList(nodeDescriptions); | ||
} | ||
|
||
@Override | ||
public String toString() { | ||
return "BootstrapConfiguration{" + | ||
"nodeDescriptions=" + nodeDescriptions + | ||
'}'; | ||
} | ||
|
||
@Override | ||
public boolean equals(Object o) { | ||
if (this == o) return true; | ||
if (o == null || getClass() != o.getClass()) return false; | ||
BootstrapConfiguration that = (BootstrapConfiguration) o; | ||
return Objects.equals(nodeDescriptions, that.nodeDescriptions); | ||
} | ||
|
||
@Override | ||
public int hashCode() { | ||
return Objects.hash(nodeDescriptions); | ||
} | ||
|
||
public static class NodeDescription implements Writeable { | ||
|
||
@Nullable | ||
private final String id; | ||
|
||
private final String name; | ||
|
||
@Nullable | ||
public String getId() { | ||
return id; | ||
} | ||
|
||
public String getName() { | ||
return name; | ||
} | ||
|
||
public NodeDescription(@Nullable String id, String name) { | ||
this.id = id; | ||
this.name = Objects.requireNonNull(name); | ||
} | ||
|
||
public NodeDescription(DiscoveryNode discoveryNode) { | ||
this(discoveryNode.getId(), discoveryNode.getName()); | ||
} | ||
|
||
public NodeDescription(StreamInput in) throws IOException { | ||
this(in.readOptionalString(), in.readString()); | ||
} | ||
|
||
@Override | ||
public void writeTo(StreamOutput out) throws IOException { | ||
out.writeOptionalString(id); | ||
out.writeString(name); | ||
} | ||
|
||
@Override | ||
public String toString() { | ||
return "NodeDescription{" + | ||
"id='" + id + '\'' + | ||
", name='" + name + '\'' + | ||
'}'; | ||
} | ||
|
||
public DiscoveryNode resolve(Iterable<DiscoveryNode> discoveredNodes) { | ||
DiscoveryNode selectedNode = null; | ||
for (final DiscoveryNode discoveredNode : discoveredNodes) { | ||
assert discoveredNode.isMasterNode() : discoveredNode; | ||
if (discoveredNode.getName().equals(name)) { | ||
if (id == null || id.equals(discoveredNode.getId())) { | ||
if (selectedNode != null) { | ||
throw new ElasticsearchException( | ||
"discovered multiple nodes matching {} in {}", this, discoveredNodes); | ||
} | ||
selectedNode = discoveredNode; | ||
} else { | ||
throw new ElasticsearchException("node id mismatch comparing {} to {}", this, discoveredNode); | ||
} | ||
} else if (id != null && id.equals(discoveredNode.getId())) { | ||
throw new ElasticsearchException("node name mismatch comparing {} to {}", this, discoveredNode); | ||
} | ||
} | ||
if (selectedNode == null) { | ||
throw new ElasticsearchException("no node matching {} found in {}", this, discoveredNodes); | ||
} | ||
|
||
return selectedNode; | ||
} | ||
|
||
@Override | ||
public boolean equals(Object o) { | ||
if (this == o) return true; | ||
if (o == null || getClass() != o.getClass()) return false; | ||
NodeDescription that = (NodeDescription) o; | ||
return Objects.equals(id, that.id) && | ||
Objects.equals(name, that.name); | ||
} | ||
|
||
@Override | ||
public int hashCode() { | ||
return Objects.hash(id, name); | ||
} | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.