Observability

Set of Jsonnet files used to deploy customized monitoring-satellites and monitoring-centrals into different clusters.

Table of contents

• Applications
  • Monitoring-satellite
  • Monitoring-central

Applications

Monitoring-satellite

Monitoring-satellite is composed by a set of components responsible for collecting and pushing observability signals from a Kubernetes cluster to a remote location (usually monitoring-central) while also being responsible for the alerting evaluation and alert routing.

Components

• Alertmanager
• Grafana
• Kube-State-Metrics
• Node-exporter
• Prometheus
• Prometheus-Operator
• OpenTelemetry-Collector (If tracing support is enabled)

To customize the stack, we make use of Jsonnet's external-variables feature. We expect one single external variable called config which is loaded and merged with monitoring-satellite to customize the stack.

We expect config to be a JSON object, where extra configuration can be added as we develop new features for monitoring-satellite. For more details, please check monitoring-satellite data schema.

A minimal example would be:

jsonnet -c -J vendor -m monitoring-satellite/manifests \
--ext-code config="{
    namespace: 'monitoring-satellite',
    clusterName: 'fake-cluster',
}" \
monitoring-satellite/manifests/yaml-generator.jsonnet | xargs -I{} sh -c 'cat {} | gojsontoyaml > {}.yaml' -- {}

Monitoring-central

Components

• Grafana
• VictoriaMetrics

To customize the stack, we make use of Jsonnet's external-variables feature. We expect one single external variable called config which is loaded and merged with monitoring-satellite to customize the stack.

We expect config to be a JSON object, where extra configuration can be added as we develop new features for monitoring-central. For more details, please check monitoring-central data schema.

A minimal example would be:

jsonnet -c -J vendor -m monitoring-central/manifests \
--ext-code config="{
    namespace: 'monitoring-central',
    grafana: {
        nodePort: 32164,
        DNS: 'http://fake.grafana.url',
        GCPExternalIpAddress: 'fake_external_ip_address',
        IAPClientID: 'fakeIAP_ID',
        IAPClientSecret: 'fakeIAP_secret',
    },
    victoriametrics: {
        DNS: 'http://fake.victoriametrics.url',
        authKey: 'random-key',
        username: 'p@ssW0rd',
        password: 'user',
        GCPExternalIpAddress: 'fake_external_ip_address',
    },
}" \
monitoring-central/manifests/yaml-generator.jsonnet | xargs -I{} sh -c 'cat {} | gojsontoyaml > {}.yaml' -- {}