[ws-manager] Make cluster selection filter by application cluster

[andrew-farries]

Description

Redo of #14000 (reverted in #14029). The original lacked the changes to components/gitpod-db/src/typeorm/workspace-cluster-db-impl.ts. This PR adds that filtering and adds tests for it.

Context

As part of #9198 we want to start syncing the `d_b_workspace_cluster` table with `db-sync`.

Currently, the table differs between US and EU regions because each table contains only the data relevant to that region. For example, in the EU table, the eu70 workspace cluster is marked as available and the us70 cluster is cordoned. In the US cluster eu70 is cordoned and us70 is available.

In order to sync the table we need to get to a point where there is no difference in the data in the table between EU and US regions.

To do that we will introduce a new field in the table called applicationCluster which records the name of the application cluster to which the record belongs. Thus, for each workspace cluster there will be two rows in Gitpod SaaS:

name	applicationCluster	url	tls	state	...
eu70	eu02	url	tls info	available	...
eu70	us02	url	tls info	cordoned	...

Effectively the new applicationCluster column gives the table an extra dimension so that we can combine both tables (EU and US) into one.

#13722 added the column to the table and made gpctl fill the value when gpctl registering a new workspace cluster. The value is taken from the GITPOD_INSTALLATION_SHORTNAME environment variable in ws-manager-bridge.

Make the cluster selection mechanism in ws-manager-api aware of the application cluster to which each workspace cluster is registered. Previously, the selection was by workspace cluster name only. Soon, as described in the context section, each region in gitpod will store records in the database for all workspace clusters - not just the ones in its region. This PR ensures that workspace clusters not in the same region as the application cluster will not be considered.

Related Issue(s)

Part of #9198 and #13800

How to test

Edit the server deployment and change the WSMAN_CFG_MANAGERS environment variable to change the applicationCluster to which the default workspace cluster in the preview environment is registered:

[
  {
    "name": "",
    "url": "dns:///ws-manager:8080",
    "tls": {
      "ca": "/ws-manager-client-tls-certs/ca.crt",
      "crt": "/ws-manager-client-tls-certs/tls.crt",
      "key": "/ws-manager-client-tls-certs/tls.key"
    },
    "state": "available",
    "maxScore": 100,
    "score": 50,
    "govern": true,
    "admissionConstraints": null,
    "applicationCluster": "dev" // <- change this from "dev" to anything else
  }
]

The WSMAN_CFG_MANAGERS environment variable is base64 encoded.

Once the server deployment is edited it should no longer be possible to start a workspace as the only workspace cluster is now associated with a different (non-existent) application cluster:

Edit the WSMAN_CFG_MANAGERS environment variable once more and set the applicationCluster field back to "dev". Starting a workspace should now proceed as normal.

Release Notes

NONE

Documentation

Werft options:

• /werft with-local-preview
  If enabled this will build install/preview
• /werft with-preview
• /werft with-integration-tests=all
  Valid options are all, workspace, webapp, ide

[werft-gitpod-dev-com]

started the job as gitpod-build-af-make-cluster-selection-app-cluster-aware.2 because the annotations in the pull request description changed
(with .werft/ from main)

[andrew-farries]

I plan to close this draft in favour of a series of smaller PRs using a DI approach, stating with the dependency injection PR: #14084

I'll address your comments here in each of those PRs @geropl.

[geropl]

@andrew-farries Why not pass the applicationCluster in parallel to name? Then there is no need for DI, at all, no?

[andrew-farries]

@andrew-farries Why not pass the applicationCluster in parallel to name? Then there is no need for DI, at all, no?

@geropl The approach we synced about is implemented in 37eb088; passing the application cluster explicitly everywhere rather than injecting a new dependency into WorkspaceManagerClientProvider (as in #14084).

This approach results in more changes across the codebase, but it's more explicit.

Still not sure which I prefer tbh.

[andrew-farries]

taking this out of draft as I think we are past the 'use DI/don't use DI' discussion now.

[geropl]

Code looks good, starting to test now.

[geropl]

Code LGTM, tested and works as expected. Thx @andrew-farries !

[geropl]

@andrew-farries Are we waiting for another review? Or can we start rolling it out?

[andrew-farries]

It's waiting for review from @aledbf or @sagor999