Adding GEP 922: Gateway API Versioning Plan #923

robscott · 2021-10-29T18:50:26Z

What type of PR is this?
/kind gep

What this PR does / why we need it:
This proposes a set of versioning guidelines for future Gateway API releases. This is a follow up to the related doc that has been discussed in gateway-api and sig-apimachinery meetings.

Which issue(s) this PR fixes:
Related to #922, will not be fixed until this GEP moves to "Implemented"

Does this PR introduce a user-facing change?:

NONE

k8s-ci-robot · 2021-10-29T18:50:42Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: robscott

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [robscott]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

youngnick

This is looking pretty good overall to me, but there's a couple of things that could be clearer.

site-src/geps/gep-922.md

hbagdi · 2021-11-02T17:37:23Z

site-src/geps/gep-922.md

+```
+gateway.networking.k8s.io/bundle-version: v0.4.0
+gateway.networking.k8s.io/channel: stable|experimental
+```


So, at the end of the day, a cluster can either have a stable or an experimental CRD. It is exclusive because you can only have one CRD at a time for a GK.

Why have the distinction in that case?
What if we just ship the experimental fields as part of the stable channel?

If we do that, then the only way for YAML consumers (and let's be real, most people will use YAML, not read the full spec) to know what fields are experimental is for us to prefix them somehow - that is, we'd have to call all the experimental fields experimentalFieldName or something.

Other things I can think of that only supplying on CRD spec would imply:

Implementations would need a switch of some sort to turn on experimental field processing - effectively, we're moving the feature gate to the implementing controller, and trusting them to figure it out.

We'd also need to supply a method for implementations to determine what the experimental fields are, so that they can tell people they're using an experimental field but haven't enabled it.

Having two separate CRDs solves all of those problems - the implementation can switch experimental field processing on or off based on the annotation on the CRD object itself (this also means you can't enable experimental field processing without having modify CRDs access, which I think is fine).

Agree with @youngnick's points above. I think it's important to have users explicitly opt in to experimental functionality, and I think separate CRDs are likely the best mechanism we have for this. Although I'd really like something better like kubectl support/gating for alpha fields, I don't think that's going to be practical anytime soon.

site-src/geps/gep-922.md

hbagdi · 2021-11-08T17:55:06Z

site-src/geps/gep-922.md

+    * Making required fields optional
+    * Removal of experimental fields
+    * Removal of experimental resources
+    * Graduation of fields or resources from experimental to stable track


I totally see the need for doing this within the same API version but I also think it is our responsibility to help out implementers here. Is there any way a controller can check the version bundle version in the cluster?
Like an annotation on the CRD (I'm not sure if those annotations are readable programmatically from a controller or not).

Since Gateways are very sensitive to security, most operators will prefer a hard failure compared to an implementation having buggy behavior (or a vulnerable behavior). We don't need to instruct what controller authors should do when the bungle version is higher than the controller is programmed against.

Can we make this part of this GEP?

Good point - we do have annotations proposed further down that would enable this. Annotations should be able to be read programmatically, as long as the controller requests read access on CRDs which seems reasonable and likely necessary.

site-src/geps/gep-922.md

hbagdi · 2021-11-08T18:00:03Z

site-src/geps/gep-922.md

+Note that each new bundle version, no matter how small, may include updated
+CRDs, webhook, or both.
+
+### Implementers:


What if an implementation doesn't want to work in a cluster unless the installed version matches a support matrix that the it is programmed against? Is that invalid behavior?

I think that's valid, but likely unnecessarily constrained. If all the guidelines here are followed, newer additions to the API would simply not be supported by an older implementation of the API. A warning or set of warnings is likely preferable to stopping altogether.

Okay. Can we explicitly mention that controllers can rely on the annotation to decide their behavior?

site-src/geps/gep-922.md

youngnick

This lgtm with a few non-blocking nits.

youngnick · 2021-11-10T23:22:47Z

site-src/geps/gep-922.md

+the following:
+
+* API Types/CRDs
+* Validating Webhook


Should we mention other validation code here?

I think we should definitely add that to our bundle versions if/when we have it. I have a note in #926 to come back to this KEP and update it if/when we add that logic.

youngnick · 2021-11-10T23:23:48Z

site-src/geps/gep-922.md

+responsibilities to ensure we're providing a consistent experience.
+
+### API Authors:
+* MUST provide conversion between stable API versions, starting with v1alpha2.


Suggested change

* MUST provide conversion between stable API versions, starting with v1alpha2.

* MUST provide conversion between API versions, starting with v1alpha2.

Feels weird to call an alpha API "stable".

Yeah, the terminology here is confusing. I've updated this to "excluding experimental fields", which I think is a clearer representation of what I was aiming for here.

youngnick · 2021-11-10T23:25:41Z

site-src/geps/gep-922.md

+  crashing
+* MUST NOT rely on webhook or CRD validation as a security mechanism. If field
+  values need to be escaped to secure an implementation, both webhook and CRD
+  validation can be bypassed and cannot be relied on. Instead, implementations


This wording is fine for now, but if we do implement the validation module, feels like it should be a MUST to me.

Good catch, I added a note to #926 so we remember to come back to this GEP and update it.

youngnick · 2021-11-10T23:28:12Z

/lgtm
/hold for multiple lgtm though.

hbagdi · 2021-11-11T23:56:05Z

Here is to hoping we have thought this through.
/lgtm

robscott · 2021-11-15T05:03:56Z

I think there's broad enough consensus on this to merge it in, we can revisit through smaller tweaks if needed.

/hold cancel

k8s-ci-robot added kind/gep PRs related to Gateway Enhancement Proposal(GEP) cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 29, 2021

k8s-ci-robot requested review from jpeach and youngnick October 29, 2021 18:50

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 29, 2021

youngnick reviewed Nov 1, 2021

View reviewed changes

site-src/geps/gep-922.md Show resolved Hide resolved

site-src/geps/gep-922.md Show resolved Hide resolved

robscott changed the title ~~Adding GEP 922: Adding Gateway API Versioning~~ Adding GEP 922: Gateway API Versioning Plan Nov 2, 2021

robscott force-pushed the gep-922 branch from b8a84b9 to 6c4c497 Compare November 2, 2021 01:17

hbagdi reviewed Nov 2, 2021

View reviewed changes

bowei reviewed Nov 3, 2021

View reviewed changes

site-src/geps/gep-922.md Show resolved Hide resolved

robscott force-pushed the gep-922 branch 2 times, most recently from cd82f80 to eab8df7 Compare November 5, 2021 23:50

hbagdi mentioned this pull request Nov 8, 2021

Summary of out-of-tree problems #629

Closed

hbagdi reviewed Nov 8, 2021

View reviewed changes

robscott mentioned this pull request Nov 8, 2021

Develop validation package that can be imported by implementations #926

Closed

robscott force-pushed the gep-922 branch 2 times, most recently from 6f26f5e to 2e138a1 Compare November 10, 2021 22:07

robscott added this to the v1beta1 milestone Nov 10, 2021

youngnick reviewed Nov 10, 2021

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 10, 2021

k8s-ci-robot assigned youngnick Nov 10, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 10, 2021

Adding GEP 922: Adding Gateway API Versioning

6f4790a

robscott force-pushed the gep-922 branch from 2e138a1 to 6f4790a Compare November 11, 2021 00:08

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 11, 2021

k8s-ci-robot assigned hbagdi Nov 11, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 11, 2021

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 15, 2021

k8s-ci-robot merged commit 01ecd5a into kubernetes-sigs:master Nov 15, 2021

robscott deleted the gep-922 branch January 8, 2022 01:05

youngnick mentioned this pull request Jan 28, 2022

GEP: Gateway API Versioning #922

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding GEP 922: Gateway API Versioning Plan #923

Adding GEP 922: Gateway API Versioning Plan #923

robscott commented Oct 29, 2021

k8s-ci-robot commented Oct 29, 2021

youngnick left a comment

hbagdi Nov 2, 2021

youngnick Nov 3, 2021

robscott Nov 5, 2021

hbagdi Nov 8, 2021

robscott Nov 8, 2021

hbagdi Nov 8, 2021

robscott Nov 8, 2021

hbagdi Nov 10, 2021

youngnick left a comment

youngnick Nov 10, 2021

robscott Nov 11, 2021

youngnick Nov 10, 2021

robscott Nov 11, 2021

youngnick Nov 10, 2021

robscott Nov 11, 2021

youngnick commented Nov 10, 2021

hbagdi commented Nov 11, 2021

robscott commented Nov 15, 2021

	* MUST provide conversion between stable API versions, starting with v1alpha2.
	* MUST provide conversion between API versions, starting with v1alpha2.

Adding GEP 922: Gateway API Versioning Plan #923

Adding GEP 922: Gateway API Versioning Plan #923

Conversation

robscott commented Oct 29, 2021

k8s-ci-robot commented Oct 29, 2021

youngnick left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

youngnick left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

youngnick commented Nov 10, 2021

hbagdi commented Nov 11, 2021

robscott commented Nov 15, 2021