Skip to content

Move file-based discovery to core #33241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,23 @@
# mechanism. This file should contain one entry per line, where an entry is a
# host/port combination. The host and port should be separated by a `:`. If
# the port is left off, a default port of 9300 is assumed. For example, if the
# cluster has three nodes that participate in the discovery process:
# (1) 66.77.88.99 running on port 9300 (2) 66.77.88.100 running on port 9305
# and (3) 66.77.88.99 running on port 10005, then this file should contain the
# cluster has three nodes that participate in the discovery process: (1)
# 66.77.88.99 running on port 9300 (2) 66.77.88.100 running on port 9305 and
# (3) 66.77.88.99 running on port 10005, then this file should contain the
# following text:
#
#10.10.10.5
#10.10.10.6:9305
#10.10.10.5:10005
#
# For IPv6 addresses, make sure to put a bracket around the host part of the address,
# for example: [2001:cdba:0000:0000:0000:0000:3257:9652]:9301 (where 9301 is the port).
# For IPv6 addresses, make sure to put a bracket around the host part of the
# address, for example: [2001:cdba:0000:0000:0000:0000:3257:9652]:9301 (where
# 9301 is the port).
#
# NOTE: all lines starting with a `#` are comments, and comments must exist on
# lines of their own (i.e. comments cannot begin in the middle of a line)
#
# NOTE: Set `discovery.zen.hosts_provider: file` to enable file-based discovery
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB this file is now included in all distributions, rather than just in the plugin. The only change, apart from reformatting, is this NOTE.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted this and gone back to having this file installed by the plugin. It cannot simply be removed because of BWC: for instance the following command would break if the directory were not there.

some-clever-stuff.sh > $ES_PATH_CONF/discovery-file/unicast_hosts.txt

# and use this file. If this setting is not set then Elasticsearch will not
# read this file.
#
# NOTE: all lines starting with a `#` are comments, and comments must exist
# on lines of their own (i.e. comments cannot begin in the middle of a line)
#
69 changes: 4 additions & 65 deletions docs/plugins/discovery-file.asciidoc
Original file line number Diff line number Diff line change
@@ -1,71 +1,10 @@
[[discovery-file]]
=== File-Based Discovery Plugin

The file-based discovery plugin uses a list of hosts/ports in a `unicast_hosts.txt` file
in the `config/discovery-file` directory for unicast discovery.
The functionality provided by the `discovery-file` plugin is now available in
Elasticsearch without requiring a plugin. This plugin still exists for backwards
compatibility, but it no longer does anything and will be removed in a future
version.

:plugin_name: discovery-file
include::install_remove.asciidoc[]

[[discovery-file-usage]]
[float]
==== Using the file-based discovery plugin

The file-based discovery plugin provides the ability to specify the
unicast hosts list through a simple `unicast_hosts.txt` file that can
be dynamically updated at any time. To enable, add the following in `elasticsearch.yml`:

[source,yaml]
----
discovery.zen.hosts_provider: file
----

This plugin simply provides a facility to supply the unicast hosts list for
zen discovery through an external file that can be updated at any time by a side process.

For example, this gives a convenient mechanism for an Elasticsearch instance
that is run in docker containers to be dynamically supplied a list of IP
addresses to connect to for zen discovery when those IP addresses may not be
known at node startup.

Note that the file-based discovery plugin is meant to augment the unicast
hosts list in `elasticsearch.yml` (if specified), not replace it. Therefore,
if there are valid unicast host entries in `discovery.zen.ping.unicast.hosts`,
they will be used in addition to those supplied in `unicast_hosts.txt`.

Anytime a change is made to the `unicast_hosts.txt` file, even as Elasticsearch
continues to run, the new changes will be picked up by the plugin and the
new hosts list will be used for the next pinging round for master election.

Upon installation of the plugin, a default `unicast_hosts.txt` file will
be found in the `$CONFIG_DIR/discovery-file` directory. This default file
will contain some comments about what the file should contain. All comments
for this file must appear on their lines starting with `#` (i.e. comments
cannot start in the middle of a line).

[[discovery-file-format]]
[float]
==== unicast_hosts.txt file format

The format of the file is to specify one unicast host entry per line.
Each unicast host entry consists of the host (host name or IP address) and
an optional transport port number. If the port number is specified, is must
come immediately after the host (on the same line) separated by a `:`.
If the port number is not specified, a default value of 9300 is used.

For example, this is an example of `unicast_hosts.txt` for a cluster with
four nodes that participate in unicast discovery, some of which are not
running on the default port:

[source,txt]
----------------------------------------------------------------
10.10.10.5
10.10.10.6:9305
10.10.10.5:10005
# an IPv6 address
[2001:0db8:85a3:0000:0000:8a2e:0370:7334]:9301
----------------------------------------------------------------

Host names are allowed instead of IP addresses (similar to
`discovery.zen.ping.unicast.hosts`), and IPv6 addresses must be
specified in brackets with the port coming after the brackets.
Loading