🌱 Attempt older version upgrades twice to work around flake with the docker controller

[cprivitere]

What this PR does / why we need it: This is a recommendation from this comment #11610 (comment) in #11610 to work around flaky upgrade tests and improve our CI signal. The root cause seems to be related to the docker controller but only affects older versions. So the goal is to just reduce the flakey signal by retrying the tests on those older versions (which just show up in our upgrade tests).

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #11209
Fixes #11610

/area e2e-testing

[cprivitere]

cc @chrischdi @sbueringer

[chrischdi]

Thanks!

/lgtm

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: d6a7cb7b308a1d865df535395ec61baa0f1b541d

[sbueringer]

Thank you!

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sbueringer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [sbueringer]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[cprivitere]

/cherry-pick release-1.9

[k8s-infra-cherrypick-robot]

@cprivitere: new pull request created: #11793

In response to this:

/cherry-pick release-1.9

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[cprivitere]

/cherry-pick release-1.8

[k8s-infra-cherrypick-robot]

@cprivitere: #11759 failed to apply on top of branch "release-1.8":

Applying: ci: Attempt older version upgrades twice to work around flake with the docker controller
Using index info to reconstruct a base tree...
M	test/e2e/clusterctl_upgrade_test.go
Falling back to patching base and 3-way merge...
Auto-merging test/e2e/clusterctl_upgrade_test.go
CONFLICT (content): Merge conflict in test/e2e/clusterctl_upgrade_test.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 ci: Attempt older version upgrades twice to work around flake with the docker controller

In response to this:

/cherry-pick release-1.8

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.