Bind host all instead of just _site_ when needed (#83145)

For security on by default we changed the interface where
elasticsearch binds by default to be:
- _site_ for the HTTP layer so that we have better chances that
kibana can communicate to elasticsearch by default as they might
not be running on the same host
- _site_ for the transport layer when we could determine that
there are other existing nodes of this cluster on different nodes.

This commit changes the behavior so that we bind to 0.0.0.0 in
these cases for the following reasons:

- We don't expect hosts nowdays to have publicly routable IP
addresses attached to any of their interfaces, and expect that
the have a site local address behind some proxy/router. In that
respect, binding to 0.0.0.0 doesn't expose elasticsearch to a
greater network segment than binding to the site local address
would
- It is significantly easier to document, explain and argue about
with users in the comments and our documentation.

Author: jkakavas

docs/changelog/83145.yaml

@@ -0,0 +1,5 @@
+pr: 83145
+summary: Bind host all instead of just _site_ when needed
+area: Security
+type: enhancement
+issues: []

qa/os/src/test/java/org/elasticsearch/packaging/test/PackagingTestCase.java

@@ -671,7 +671,7 @@ public void verifySecurityAutoConfigured(Installation es) throws Exception {
         assertThat(settings.get("xpack.security.enabled"), equalTo("true"));
 
         if (es.distribution.isDocker() == false) {
-            assertThat(settings.get("http.host"), equalTo("[_local_, _site_]"));
+            assertThat(settings.get("http.host"), equalTo("0.0.0.0"));
         }
     }
 

x-pack/docs/en/security/enroll-nodes.asciidoc

@@ -1,8 +1,8 @@
 [role="exclude"]
 
 When {es} starts for the first time, the security auto-configuration process
-binds the HTTP layer to both `_site_` and `_local_`, but only binds the
-transport layer to `_local_`. This intended behavior ensures that you can start
+binds the HTTP layer to `0.0.0.0`, but only binds the transport layer to
+localhost. This intended behavior ensures that you can start
 a single-node cluster with security enabled by default without any additional
 configuration.
 
@@ -14,10 +14,10 @@ could expire, which is why enrollment tokens aren't generated automatically.
 Additionally, only nodes on the same host can join the cluster without
 additional configuration. If you want nodes from another host to join your
 cluster, you need to set `transport.host` to a
-{ref}/modules-network.html#network-interface-values[supported value] other than
-`_local_` (such as `_site_`), or an IP address that's bound to an interface
-where other hosts can reach it. Refer to 
-{ref}/modules-network.html#transport-settings[transport settings] for more 
+{ref}/modules-network.html#network-interface-values[supported value]
+(such as uncommenting the suggested value of `0.0.0.0`), or an IP address
+that's bound to an interface where other hosts can reach it. Refer to
+{ref}/modules-network.html#transport-settings[transport settings] for more
 information.
 
 To enroll new nodes in your cluster, create an enrollment token with the

x-pack/plugin/security/cli/src/main/java/org/elasticsearch/xpack/security/cli/AutoConfigureNode.java

@@ -782,13 +782,11 @@ protected void execute(Terminal terminal, OptionSet options, Environment env) th
                         || localFinalEnv.settings().hasValue(NetworkService.GLOBAL_NETWORK_BIND_HOST_SETTING.getKey())
                         || localFinalEnv.settings().hasValue(NetworkService.GLOBAL_NETWORK_PUBLISH_HOST_SETTING.getKey()))) {
                         bw.newLine();
-                        bw.write("# Allow HTTP API connections from localhost and local networks");
+                        bw.write("# Allow HTTP API connections from anywhere");
                         bw.newLine();
                         bw.write("# Connections are encrypted and require user authentication");
                         bw.newLine();
-                        bw.write(
-                            HttpTransportSettings.SETTING_HTTP_HOST.getKey() + ": " + hostSettingValue(NetworkUtils.getAllAddresses())
-                        );
+                        bw.write(HttpTransportSettings.SETTING_HTTP_HOST.getKey() + ": 0.0.0.0");
                         bw.newLine();
                     }
                     if (false == (localFinalEnv.settings().hasValue(TransportSettings.HOST.getKey())
@@ -798,15 +796,15 @@ protected void execute(Terminal terminal, OptionSet options, Environment env) th
                         || localFinalEnv.settings().hasValue(NetworkService.GLOBAL_NETWORK_BIND_HOST_SETTING.getKey())
                         || localFinalEnv.settings().hasValue(NetworkService.GLOBAL_NETWORK_PUBLISH_HOST_SETTING.getKey()))) {
                         bw.newLine();
-                        bw.write("# Allow other nodes to join the cluster from localhost and local networks");
+                        bw.write("# Allow other nodes to join the cluster from anywhere");
                         bw.newLine();
                         bw.write("# Connections are encrypted and mutually authenticated");
                         bw.newLine();
                         if (false == inEnrollmentMode
                             || false == anyRemoteHostNodeAddress(transportAddresses, NetworkUtils.getAllAddresses())) {
                             bw.write("#");
                         }
-                        bw.write(TransportSettings.HOST.getKey() + ": " + hostSettingValue(NetworkUtils.getAllAddresses()));
+                        bw.write(TransportSettings.HOST.getKey() + ": 0.0.0.0");
                         bw.newLine();
                     }
                     bw.newLine();
@@ -880,14 +878,6 @@ protected static boolean anyRemoteHostNodeAddress(List<String> allNodesTransport
         return false;
     }
 
-    protected String hostSettingValue(InetAddress[] allAddresses) {
-        if (Arrays.stream(allAddresses).anyMatch(InetAddress::isSiteLocalAddress)) {
-            return "[_local_, _site_]";
-        } else {
-            return "[_local_]";
-        }
-    }
-
     private Environment possiblyReconfigureNode(Environment env, Terminal terminal) throws UserException {
         // We remove the existing auto-configuration stanza from elasticsearch.yml, the elastisearch.keystore and
         // the directory with the auto-configured TLS key material, and then proceed as if elasticsearch is started