Merge pull request #1744 from youngnick/gep-1742-timeouts

GEP: Add Timeouts GEP first cut

Author: k8s-ci-robot

mkdocs.yml

@@ -24,6 +24,7 @@ plugins:
       redirect_maps:
         'guides/getting-started.md': 'guides/index.md'
         'contributing/community.md': 'contributing/index.md'
+  - mermaid2
 markdown_extensions:
   - admonition
   - meta
@@ -32,10 +33,15 @@ markdown_extensions:
       emoji_generator: !!python/name:materialx.emoji.to_svg
   - pymdownx.highlight
   - pymdownx.inlinehilite
-  - pymdownx.superfences
+  - pymdownx.superfences:
+      custom_fences:
+        - name: mermaid
+          class: mermaid
+          format: !!python/name:pymdownx.superfences.fence_code_format
   - pymdownx.snippets
   - toc:
       permalink: true
+  - tables
 nav:
   - Overview:
     - Introduction: index.md
@@ -72,6 +78,7 @@ nav:
       - Declined:
         - geps/gep-735.md
       - Provisional:
+        - geps/gep-1742.md
         - geps/gep-1426.md
         - geps/gep-1324.md
         - geps/gep-1282.md

requirements.txt

@@ -13,6 +13,7 @@ mkdocs-macros-plugin==0.7.0
 mkdocs-material==9.1.1
 mkdocs-material-extensions==1.1.1
 mkdocs-redirects==1.2.0
+mkdocs-mermaid2-plugin==0.6.0
 pep562==1.1
 Pygments==2.14.0
 pymdown-extensions==9.10

site-src/geps/gep-1742.md

@@ -0,0 +1,315 @@
+# GEP-1742: Timeouts
+
+* Issue: [#1742](https://github.com/kubernetes-sigs/gateway-api/issues/1742)
+* Status: Provisional
+
+(See status definitions [here](overview.md#status).)
+
+## TLDR
+
+Create some sort of design so that Gateway API objects can be used to configure
+timeouts for different types of connection.
+
+## Goals
+
+- Create some method to configure some timeouts
+- Timeout config must be applicable to most if not all Gateway API implementations.
+
+## Non-Goals
+
+- TBD
+
+## Introduction
+
+In talking about Gateway API objects, particularly HTTPRoute, we've mentioned
+timeout configuration many times in the past as "too hard" to find the common
+ground necessary to make more generic configuration. This GEP intends firstly
+to make this process less difficult, then to find common timeouts that we can
+build into Gateway API.
+
+For this initial round, we'll focus on Layer 7 HTTP traffic, while acknowledging
+that Layer 4 connections have their own interesting timeouts as well.
+
+The following sections will review all the implementations, then document what
+timeouts are _available_ for the various data planes.
+
+### Background on implementations
+
+Most implementations that handle HTTPRoute objects use a proxy as the data plane
+implementation, that actually forwards flows as directed by Gateway API configuration.
+
+The following table is a review of all the listed implementations of Gateway API
+at the time of writing, with the data plane they use for Layer 7, based on what information
+could be found online. If there are errors here, or if the implementation doesn't
+support layer 7, please feel free to correct them.
+
+| Implementation | Data Plane       |
+|----------------|------------|
+| Acnodal EPIC   | Envoy      |
+| Apache APISIX  | Nginx      |
+| BIG-IP Kubernetes Gateway| F5 BIG-IP  |
+| Cilium         | Envoy      |
+| Contour        | Envoy      |
+| Emissary Ingress| Envoy     |
+| Envoy Gateway  | Envoy      |
+| Flomesh Service Mesh | Pipy |
+| Gloo Edge      | Envoy      |
+| Google Kubernetes Engine (GKE) | Similar to Envoy Timeouts |
+| HAProxy Ingress | HAProxy   |
+| Hashicorp Consul | Envoy    |
+| Istio          | Envoy      |
+| Kong           | Nginx      |
+| Kuma           | Envoy      |
+| Litespeed      | Litespeed WebADC |
+| Nginx Kubernetes Gateway | Nginx |
+| Traefik        | Traefik    |
+
+
+### Flow diagrams with available timeouts
+
+The following flow diagrams are based off the basic diagram below, with all the
+timeouts I could find included.
+
+In general, timeouts are recorded with the setting name or similar that the data
+plane uses for them, and are correct as far as I've parsed the documentation
+correctly.
+
+Idle timeouts are marked as such.
+
+```mermaid
+sequenceDiagram
+    participant C as Client
+    participant P as Proxy
+    participant U as Upstream
+    C->>P: Connection Started
+    C->>P: Starts sending Request
+    C->>P: Finishes Headers
+    C->>P: Finishes request
+    P->>U: Connection Started
+    P->>U: Starts sending Request
+    P->>U: Finishes request
+    P->>U: Finishes Headers
+    U->>P: Starts Response
+    U->>P: Finishes Headers
+    U->>P: Finishes Response
+    P->>C: Starts Response
+    P->>C: Finishes Headers
+    P->>C: Finishes Response
+    Note right of P: Repeat if connection sharing
+    U->>C: Connection ended
+```
+
+#### Envoy Timeouts
+
+For Envoy, some timeouts are configurable at either the HTTP Connection Manager
+(very, very roughly equivalent to a Listener), the Route (equivalent to a HTTPRoute)
+level, or the Cluster (usually close to the Service) or some combination. These
+are noted in the below diagram with a `CM`, `R`, or `Cluster` prefix respectively.
+
+```mermaid
+sequenceDiagram
+    participant C as Client
+    participant P as Envoy
+    participant U as Upstream
+    C->>P: Connection Started
+    activate P
+    Note left of P: transport_socket_connect_timeout for TLS
+    deactivate P
+    C->>P: Starts sending Request
+    activate C
+    activate P
+    activate P
+    C->>P: Finishes Headers
+    note left of P: CM request_headers_timeout
+		C->>P: Finishes request
+    deactivate P
+    activate U
+    note left of U: Cluster connect_timeout
+    deactivate U
+    P->>U: Connection Started
+		activate U
+    note right of U: CM idle_timeout<br />CM max_connection_duration
+    P->>U: Starts sending Request
+    P->>U: Finishes Headers
+    note left of P: CM request_timeout
+		P->>U: Finishes request
+    deactivate P
+    activate U
+    U->>P: Starts Response
+    U->>P: Finishes Headers
+		note right of U: R timeout<br/>R per_try_timeout<br/>R per_try_idle_timeout
+    U->>P: Finishes Response
+    deactivate U
+    P->>C: Starts Response
+    P->>C: Finishes Headers
+    P->>C: Finishes Response
+    Note left of C: CM stream_idle_timeout<br />R idle_timeout<br />CM,R max_stream_duration<br/>TCP proxy idle_timeout<br />TCP protocol idle_timeout
+    deactivate C
+    Note right of P: Repeat if connection sharing
+    U->>C: Connection ended
+    deactivate U
+```
+
+#### Nginx timeouts
+
+Nginx allows setting of GRPC and general HTTP timeouts separately, although the
+purposes seem to be roughly equivalent.
+
+```mermaid
+sequenceDiagram
+    participant C as Client
+    participant P as Nginx
+    participant U as Upstream
+    C->>P: Connection Started
+    C->>P: Starts sending Request
+    C->>P: Finishes Headers
+    C->>P: Finishes request
+    P->>U: Connection Started
+    Activate U
+	  Activate U
+    P->>U: Starts sending Request
+    activate U
+    note left of U: proxy_connect_timeout<br/>grpc_connect_timeout
+    deactivate U
+    P->>U: Finishes Headers
+		P->>U: Finishes request
+    Note right of U: (between write operations)<br/>proxy_send_timeout<br/>grpc_send_timeout
+    deactivate U
+		activate U
+    U->>P: Starts Response
+    U->>P: Finishes Headers
+        Note right of U: proxy_read_timeout<br/>grpc_read_timeout
+    U->>P: Finishes Response
+    deactivate U
+    P->>C: Starts Response
+    P->>C: Finishes Headers
+    P->>C: Finishes Response
+    Note right of P: Repeat if connection sharing
+    Note Right of U: keepalive_timeout (if keepalive enabled)
+    U->>C: Connection ended
+		deactivate U
+```
+
+#### HAProxy timeouts
+
+```mermaid
+sequenceDiagram
+    participant C as Client
+    participant P as Proxy
+    participant U as Upstream
+    
+    C->>P: Connection Started
+    activate U
+    activate C
+    activate P
+    note left of P: timeout client (idle)
+    C->>P: Starts sending Request
+    C->>P: Finishes Headers
+    C->>P: Finishes request
+    note left of C: timeout http-request
+    deactivate C
+			activate C
+    note left of C: timeout client-fin
+    deactivate C
+		deactivate P
+    activate U
+    note left of U: timeout queue<br/>(wait for available server)
+    deactivate U
+		
+    P->>U: Connection Started
+    activate U
+    P->>U: Starts sending Request
+    activate U
+    P->>U: Finishes Headers
+    P->>U: Finishes request
+
+    note right of U: timeout connect
+    deactivate U
+    note left of U: timeout server<br/>(idle timeout)
+    deactivate U
+    activate U
+    note left of U: timeout server-fin
+    deactivate U
+    U->>P: Starts Response
+    U->>P: Finishes Headers
+    U->>P: Finishes Response
+    P->>C: Starts Response
+    P->>C: Finishes Headers
+    P->>C: Finishes Response
+    activate C
+    note left of C: timeout http-keep-alive
+    deactivate C
+    Note right of P: Repeat if connection sharing
+    Note right of U: timeout tunnel<br/>(for upgraded connections)
+    deactivate U
+    U->>C: Connection ended
+
+```
+
+#### Traefik timeouts
+
+```mermaid
+sequenceDiagram
+    participant C as Client
+    participant P as Proxy
+    participant U as Upstream
+    C->>P: Connection Started
+    activate U
+    C->>P: Starts sending Request
+    activate P
+    C->>P: Finishes Headers
+    Note right of P: respondingTimeouts<br/>readTimeout
+    C->>P: Finishes request
+    deactivate P
+    P->>U: Connection Started
+    activate U
+    Note right of U: forwardingTimeouts<br/>dialTimeout
+    deactivate U
+    P->>U: Starts sending Request
+    P->>U: Finishes request
+    P->>U: Finishes Headers
+    U->>P: Starts Response
+    activate U
+    note right of U: forwardingTimeouts<br/>responseHeaderTimeout
+    U->>P: Finishes Headers
+    deactivate U
+    U->>P: Finishes Response
+    P->>C: Starts Response
+    activate P
+    P->>C: Finishes Headers
+    Note right of P: respondingTimeouts<br/>writeTimeout
+    P->>C: Finishes Response
+    deactivate P
+    Note right of P: Repeat if connection sharing
+    Note right of U: respondingTimeouts<br/>idleTimeout<br/>Keepalive connections only
+    deactivate U
+    U->>C: Connection ended
+
+```
+#### F5 BIG-IP Timeouts
+
+Could not find any HTTP specific timeouts. PRs welcomed. 😊
+
+#### Pipy Timeouts
+
+Could not find any HTTP specific timeouts. PRs welcomed. 😊
+
+#### Litespeed WebADC Timeouts
+
+Could not find any HTTP specific timeouts. PRs welcomed. 😊
+
+
+## API
+
+TBD.
+
+
+## Alternatives
+
+(List other design alternatives and why we did not go in that
+direction)
+
+## References
+
+(Add any additional document links. Again, we should try to avoid
+too much content not in version control to avoid broken links)