fix: Don't try watching items with missing resourceVersion

[ShutterQuick]

Watching fails for resources missing a resourceVersion.
The watch is then retried again a second later, ad infinitum.

This leads to extra resource use, and an obscene amount of noise in the logs.
When resourceVersion is missing, this is normally because it's not a real object, but rather an interface to get some information from an api extension - e.g. Antrea uses this pattern, as shown in the example log below.
This issue appears to be the same thing: argoproj/argo-cd#21092

This merge request changes the behaviour for missing resourceVersion to wait for the next resync interval before trying again.
I figure this should be fine as these resources are already not working.

If you have any concerns or would like me to go about this another way (e.g. by implementing backoff instead), let me know and I'll update the MR.

Example of log illustrating the issue

The following is a three second excerpt from a cluster with multiple resources affected by this.

time="2025-03-14T09:17:47Z" level=info msg="Start watch EgressGroup.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:47Z" level=info msg="Failed to watch EgressGroup.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:47Z" level=info msg="Start watch SupportBundleCollection.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:47Z" level=info msg="Failed to watch SupportBundleCollection.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:47Z" level=info msg="Start watch AppliedToGroup.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:47Z" level=info msg="Failed to watch AppliedToGroup.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:47Z" level=info msg="Start watch NetworkPolicy.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:47Z" level=info msg="Failed to watch NetworkPolicy.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:47Z" level=info msg="Start watch AddressGroup.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:47Z" level=info msg="Failed to watch AddressGroup.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Start watch EgressGroup.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Failed to watch EgressGroup.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Start watch SupportBundleCollection.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Failed to watch SupportBundleCollection.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Start watch AppliedToGroup.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Failed to watch AppliedToGroup.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Start watch NetworkPolicy.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Failed to watch NetworkPolicy.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Start watch AddressGroup.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:48Z" level=info msg="Failed to watch AddressGroup.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=debug msg="Ignoring change of object because none of the watched resource fields have changed" api-version=external-secrets.io/v1beta1 kind=ExternalSecret name=dev-starfleet-portal-envs namespace=team-starfleet server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Start watch EgressGroup.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Failed to watch EgressGroup.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Start watch SupportBundleCollection.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Failed to watch SupportBundleCollection.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Start watch AppliedToGroup.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Failed to watch AppliedToGroup.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Start watch NetworkPolicy.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Failed to watch NetworkPolicy.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Start watch AddressGroup.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T09:17:49Z" level=info msg="Failed to watch AddressGroup.controlplane.antrea.io on https://k8s.example.com: initial RV \"\" is not supported due to issues with underlying WATCH, retrying in 1s" server="https://k8s.example.com"

Example of log with this MR applied

Note: Only contains lines containing Deployment and NetworkPolicy.controlplane.

time="2025-03-14T12:16:56Z" level=info msg="Start watch NetworkPolicy.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T12:16:57Z" level=info msg="Start watch Deployment.apps on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T12:16:58Z" level=info msg="Ignoring watch for NetworkPolicy.controlplane.antrea.io on https://k8s.example.com due to missing resourceVersion" server="https://k8s.example.com"
time="2025-03-14T12:26:57Z" level=info msg="Failed to watch Deployment.apps on https://k8s.example.com: Resyncing Deployment.apps on https://k8s.example.com due to timeout, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T12:26:57Z" level=info msg="Start watch Deployment.apps on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T12:26:58Z" level=info msg="Failed to watch NetworkPolicy.controlplane.antrea.io on https://k8s.example.com: Resyncing NetworkPolicy.controlplane.antrea.io on https://k8s.example.com due to timeout, retrying in 1s" server="https://k8s.example.com"
time="2025-03-14T12:26:58Z" level=info msg="Start watch NetworkPolicy.controlplane.antrea.io on https://k8s.example.com" server="https://k8s.example.com"
time="2025-03-14T12:26:58Z" level=info msg="Ignoring watch for NetworkPolicy.controlplane.antrea.io on https://k8s.example.com due to missing resourceVersion" server="https://k8s.example.com"

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 15 lines in your changes missing coverage. Please review.

Project coverage is 53.33%. Comparing base (8849c3f) to head (8d347a2).
Report is 35 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/cache/cluster.go	0.00%	14 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #704      +/-   ##
==========================================
- Coverage   54.26%   53.33%   -0.94%     
==========================================
  Files          64       64              
  Lines        6164     6433     +269     
==========================================
+ Hits         3345     3431      +86     
- Misses       2549     2726     +177     
- Partials      270      276       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[crenshaw-dev]

When resourceVersion is missing, this is normally because it's not a real object, but rather an interface to get some information from an api extension

Does this mean an aggregated API, or something different?

[ShutterQuick]

When resourceVersion is missing, this is normally because it's not a real object, but rather an interface to get some information from an api extension

Does this mean an aggregated API, or something different?

Yeah that's it, sorry for the imprecise language.
Guess I should also qualify that when I say "normally" I mean from subjective experience. That's the only cases I've seen it.

If I read the Kubernetes API Conventions document right I think it's probably safe to assume only aggregated APIs will ever return resources without resourceVersion, but I'm not certain.