Fix improper controller-runtime cache configuration

[dgrove-oss]

When AppWrappers are enabled, it is not correct to configure the controller-runtime cache with a filter that only allows services, secrets, etc with the RayCluster label to be cached. This breaks any AppWrapper that contains one of these resource kinds.

[dgrove-oss]

My understanding is that Kubernetes does not support OR for label selectors, so the only option is to disable the filtering entirely. If it is possible to do a filter with an OR of the RayCluster label and the AppWrapper label that would also be an acceptable fix.

[dgrove-oss]

@varshaprasad96 -- this misconfiguration is the root cause of the problem with AppWrappers containing Services we were describing to you a couple weeks back. At the time, we thought it was an issue in the AppWrapper operator when the AppWrapper contained multiple resources, but that turned out to be incorrect. The problem was the filtering of the controller-runtime cache by the codeflare operator which led the AppWrapper controller to get back a NotFound error when it tried to get a Service it had created that didn't have the RayCluster label.

[dgrove-oss]

rebased to pick up codeflare common fixes in #641

[varshaprasad96]

It would be helpful to add tests to ensure controller behaves as expected with this cache configuration. Can do it later in follow up.

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: varshaprasad96

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [varshaprasad96]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dgrove-oss]

It would be helpful to add tests to ensure controller behaves as expected with this cache configuration. Can do it later in follow up.

💯 -- one possibility would be to try to run the existing appwrapper controller end-to-end test suite. I can take a look at that if the idea seems reasonable.