Install monitoring-satellite with obs-installer

[vulkoingim]

Description

Related Issue(s)

Fixes https://github.com/gitpod-io/ops/issues/3585

How to test

werft run github -j .werft/build.yaml -a with-preview=true

https://werft.gitpod-dev.com/job/gitpod-custom-aa-obs-installer-ss.11 (fixed the popd issue with the last commit)

NAME                                 READY   STATUS    RESTARTS   AGE
alertmanager-main-0                  2/2     Running   0          38m
grafana-8dc9c7bb9-x5525              1/1     Running   0          2m20s
http-prober-867c755dd-drgkc          1/1     Running   0          38m
kube-state-metrics-876578749-4g8t2   3/3     Running   0          38m
kubescape-bdd7f8776-lkqlw            1/1     Running   0          38m
node-exporter-xnwlh                  2/2     Running   0          38m
prometheus-k8s-0                     2/2     Running   0          38m
prometheus-operator-7f5444cc-v5x6m   2/2     Running   0          38m
pyrra-api-7944b86bfd-swrds           1/1     Running   0          38m
pyrra-kubernetes-7dd5d8dd46-xqnzd    1/1     Running   0          38m

Release Notes

NONE

Werft options:

• /werft with-preview

[werft-gitpod-dev-com]

started the job as gitpod-build-aa-obs-installer-ss.1 because the annotations in the pull request description changed
(with .werft/ from main)

[vulkoingim]

We should replace this with go install, if this fixes the issue #12703

[ArthurSens]

Hey aleks, I couldn't really understand what problems we're solving by using shell instead of typescript here 🤔

Could you elaborate a bit?

[meysholdt]

Hey aleks, I couldn't really understand what problems we're solving by using shell instead of typescript here 🤔

I assume that's because it's easier to test: run the bash script from your workspace instead of running the full werft job. That's in the spirit of the discussion we had in the platform sync on Monday: Have reusable modules that can be executed individually rather than one looong spaghetti. With that being said, it does not need to be bash, it just needs to be decoupled from the job.

[vulkoingim]

In essence the server-side apply fixes the issue with the client responding that resources exist (as that check is done client-side).

As to why use shell, instead of typescript - there are a couple of reasons:

• As Mo said - small reusable pieces that we can execute and test individually
• Wrapping bash in typescript brings 0 benefits and makes no sense IMO
• ☝️ makes everything more difficult to read and follow - i.e. spaghetti
• While bash is not great - I would argue that it's much cleaner and easier to understand what this script does and how it does it exactly
• There is a similar script in ops that spins up monitoring-satellite in ws clusters. If the issue I referenced above is fixed - then we can have that script only in a single place (here), and reduce the amount of things we have to maintain.

[vulkoingim]

Here's the custom job: https://werft.gitpod-dev.com/job/gitpod-custom-aa-obs-installer-ss.16

/werft run

👍 started the job as gitpod-build-aa-obs-installer-ss.12
(with .werft/ from main)