yaml parser for elasticsearch.yml wrongly parsing lists

[naturalgeek]

Elasticsearch version (bin/elasticsearch --version):
7.5.2 (recent form official repo)

Plugins installed: []

JVM version (java -version):
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~16.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

OS version (uname -a if on a Unix-like system):
Linux elk0 4.15.0-1067-azure #72-Ubuntu SMP Wed Jan 15 15:07:27 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

Defining a list with unquoted children on the same level as the parent are not parsed properly

Example:

Works:

cluster.initial_master_nodes:
  - foo
  - bar

Also works:

cluster.initial_master_nodes:
  - "foo"
  - "bar"

Also works:

cluster.initial_master_nodes:
- "foo"
- "bar"

But
won't work:

cluster.initial_master_nodes:
- foo
- bar

According to my understanding, the last list is also a valid yaml syntax.
PyYaml used by Ansible e.g. in the filter "to_nice_yaml" produces exactly the last array.

Fun fact: elasticsearch's official role "ansible-elasticsearch" uses this filter to build the elasticsearch.yaml.
As a workaround for ansible users: you can use "to_yaml" in the template instead, which produces a working onliner.

Steps to reproduce:

Write a config in the described list format.

Provide logs (if relevant):

Logs are not relevant.

[danielmitterdorfer]

Can you please elaborate what you mean exactly when you say "not parsed properly" and "won't work"? Please include a clear reproduction scenario and your expected and the actual result.

Here's what I've tried meanwhile:

I started a single Elasticsearch node (version 7.5.2, OSS distribution) with the following elasticsearch.yml (which you mention as "does not work" above):

node.name: node-1
network.host: 0.0.0.0
cluster.initial_master_nodes:
- node-1
- node-2

After starting the node, I observed in the logs:

[2020-02-07T10:32:48,384][WARN ][o.e.c.c.ClusterFormationFailureHelper] [node-1] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [node-1, node-2] to bootstrap a cluster: have discovered [{node-1}{WXJPNw-TR3aZLk3-Oj3rrw}{9jGYI7KISHCEnfYH5Px7yw}{192.168.1.120}{192.168.1.120:9300}{dim}]; discovery will continue using [127.0.0.1:9300, 127.0.0.1:9301, 127.0.0.1:9302, 127.0.0.1:9303, 127.0.0.1:9304, 127.0.0.1:9305, [::1]:9300, [::1]:9301, [::1]:9302, [::1]:9303, [::1]:9304, [::1]:9305] from hosts providers and [{node-1}{WXJPNw-TR3aZLk3-Oj3rrw}{9jGYI7KISHCEnfYH5Px7yw}{192.168.1.120}{192.168.1.120:9300}{dim}] from last-known cluster state; node term 0, last-accepted version 0 in term 0

note the output: ...master-eligible nodes [node-1, node-2]. From this I conclude that this is parsed as a list (as expected).

Then I started the node again with the following elasticsarch.yml (correct as per your description above):

node.name: node-1
network.host: 0.0.0.0
cluster.initial_master_nodes:
    - node-1
    - node-2

The behavior that I have observed is identical.

Fun fact: elasticsearch's official role "ansible-elasticsearch" uses this filter to build the elasticsearch.yaml.

Can you please raise this as a separate issue in https://github.com/elastic/ansible-elasticsearch/?

According to the Ansible team, to_nice_yaml does not do any indentation so it should be written as follows:

{{ example | to_nice_yaml | indent(4, true) }}

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Settings)

[naturalgeek]

I start a completely new cluster of 3 systems, adding the list of master nodes as you did (but 3 nodes)
also setting the seed nodes in the same way.

the cluster won't be initialised because there is no master eligible node.
In the Error, the expected list of eligible master nodes (same as in your post) was only the fqdn of the local host in each node which for my undestanding is the default value and therefor it seems that the parser is not reading the array at all.
(sorry i'am unable to reinititialise a fresh cluster right now, so i can't paste you a log)

This is my config with slightly altered hostnames:

bootstrap.memory_lock: false
cluster.initial_master_nodes:
- elk0.logging0.secret0.somelocation.omg.foobar.net
- elk1.logging0.secret0.somelocation.omg.foobar.net
- elk2.logging0.secret0.somelocation.omg.foobar.net
cluster.name: secret0-elk0
discovery.seed_hosts:
- elk0.logging0.secret0.somelocation.omg.foobar.net
- elk1.logging0.secret0.somelocation.omg.foobar.net
- elk2.logging0.secret0.somelocation.omg.foobar.net
http.port: 9200
network.host: 0.0.0.0
node.data: true
node.master: true
node.name: elk0.logging0.secret0.somelocation.omg.foobar.io




#################################### Paths ####################################

# Path to directory containing configuration (this file and logging.yml):

path.data: /data0/elasticsearch

path.logs: /var/log/elasticsearch


action.auto_create_index: true

[DaveCTurner]

the expected list of eligible master nodes (same as in your post) was only the fqdn of the local host in each node which for my undestanding is the default value

No, the default value of cluster.initial_master_nodes is the empty list.

I tried the config you quoted and see the expected message in the logs that mentions all three nodes:

[2020-02-14T13:41:25,452][WARN ][o.e.c.c.ClusterFormationFailureHelper] [elk0.logging0.secret0.somelocation.omg.foobar.net] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [elk0.logging0.secret0.somelocation.omg.foobar.net, elk1.logging0.secret0.somelocation.omg.foobar.net, elk2.logging0.secret0.somelocation.omg.foobar.net] to bootstrap a cluster: have discovered [{elk0.logging0.secret0.somelocation.omg.foobar.net}{60E9M9uwToS4KjhhfIEGPQ}{AtPY8cV8QF22wTir953qMQ}{127.0.0.1}{127.0.0.1:9300}{dilm}{ml.machine_memory=17179869184, xpack.installed=true, ml.max_open_jobs=20}]; discovery will continue using [] from hosts providers and [{elk0.logging0.secret0.somelocation.omg.foobar.net}{60E9M9uwToS4KjhhfIEGPQ}{AtPY8cV8QF22wTir953qMQ}{127.0.0.1}{127.0.0.1:9300}{dilm}{ml.machine_memory=17179869184, xpack.installed=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0

I am closing this as as far as I can tell this is working as expected. If you can provide a way of reproducing what you're seeing then we can reopen this.