Add tiny load testing script

[mads-hartmann]

Description

This introduces a tiny script for running load tests against our VM-based preview environments. It prepares a base branch with an empty commit that contains /werft with-vm and then pushes as many branches as you have requested so that Werft will run the build job and thus create a VM and deploy Gitpod to it.

The base branch is based off the current branch, that means you can use this to create load tests based on whatever is configured on your branch. E.g. if we wanted to run the load test using a separate VM image than what we have on main then we'd just create a branch where we switch the VM image and then run the load script from that branch.

Related Issue(s)

Fixes https://github.com/gitpod-io/ops/issues/1215

How to test

If you just want to try it out you can use it to trigger a single VM with the following.

./dev/preview/test/load-test.sh --base-branch-name your-name-vm-load-test --start 1 --end 1

I have used this script quite a few times to run small loads tests against Harvester. You can see one analysis of running the script here (internal link).

Release Notes

NONE

Documentation

N/A

[codecov]

Codecov Report

Merging #8310 (c62a0d8) into main (0f1aaa7) will decrease coverage by 1.13%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #8310      +/-   ##
==========================================
- Coverage   12.31%   11.17%   -1.14%     
==========================================
  Files          20       18       -2     
  Lines        1161      993     -168     
==========================================
- Hits          143      111      -32     
+ Misses       1014      880     -134     
+ Partials        4        2       -2     

Flag	Coverage Δ
components-gitpod-cli-app	11.17% <ø> (ø)
components-local-app-app-darwin-amd64	?
components-local-app-app-darwin-arm64	?
components-local-app-app-linux-amd64	?
components-local-app-app-linux-arm64	?
components-local-app-app-windows-386	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
components/local-app/pkg/auth/pkce.go
components/local-app/pkg/auth/auth.go

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f1aaa7...c62a0d8. Read the comment docs.

[mads-hartmann]

/werft run

👍 started the job as gitpod-build-mads-load-test.8

[ArthurSens]

Simple yet effective 🚀

[kylos101]

/hold

This may be blocking Tide and other PRs from merging.

[kylos101]

/unhold

the queue has been processed, let's see if this auto merges now?

[ArthurSens]

It won't get merged because a required check is failing (ci/werft/build). The build needs to pass, then tide will merge it.

Not sure why prow puts this PR on the queue if a required check is failing :/

/hold

[kylos101]

Good catch @ArthurSens.

[ArthurSens]

Since this change has nothing to do with preview at all, I assume the failure is due to flakyness. Running a job without preview just to the the green lights ✅

/werft run no-preview

👍 started the job as gitpod-build-mads-load-test.11

[ArthurSens]

/unhold