Make sure that we have reasonable defaults

[ChunyiLyu]

Context
As part of the upgrade and GA effort, it will be good if we can make sure that the operator is using setting defaults when deploy rabbitmq. We should check defaults that the operator is setting for:

1. statefulSet deployment
2. rabbitmq.conf
3. any other defaults that impact the behavior of rabbitmq or how it is deployed.

This is to make sure that 1. the operator is using reasonable defaults; 2. we minimize the chance that we need to change the default behavior in the future.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[mkuratczyk]

Some settings to consider:

• disk_free_limit.relative = 1.0 (https://github.com/rabbitmq/tgir/blob/S01E06/s01/e06/k8s/configmap.disk-free-limit.yml)
• total_memory_available_override_value = 12GB (https://github.com/rabbitmq/tgir/blob/S01E06/s01/e06/k8s/configmap.total-memory-available-override-value.yml)
• review https://github.com/rabbitmq/tgir/blob/S01E06/s01/e06/k8s/statefulset.yml

[mkuratczyk]

Things to discuss:

1. Do we want to use conf.d format for the settings we generate?

2. At some point we added these settings:

cluster_formation.node_cleanup.interval         = 30
cluster_formation.node_cleanup.only_log_warning = true

Is it the right thing to do?

3. startupProbe is still being discussed Pod stuck in Terminating state #409

4. Is it really necessary to set terminationGracePeriodSeconds to such a high value?

5. Should we rename persistence given multiple volumes could be used?

6. No command set in the StatefulSet. Do we want to keep it like that?

7. Rename cluster-links to erlang in the headless service

[mkuratczyk]

We document the default configuration in https://www.rabbitmq.com/kubernetes/operator/using-operator.html#additional-config. We need to update it once we are done with default configuration changes.