Skip to content

Unhandled BaseException cause upstrream connection disconnected sliently #2923

Closed
@min-tian-thomas

Description

@min-tian-thomas

Context
If Gunicron is configured with keepalive != 0, then base_async.py will handle it with keepalive loop.

                    # keepalive loop
                    proxy_protocol_info = {}
                    while True:
                        req = None
                        with self.timeout_ctx():
                            req = next(parser)
                        if not req:
                            break
                        if req.proxy_protocol_info:
                            proxy_protocol_info = req.proxy_protocol_info
                        else:
                            req.proxy_protocol_info = proxy_protocol_info
                        self.handle_request(listener_name, req, client, addr)

Here when using worker type as: worker_class = "gevent", then gevent.py defined as:

    def timeout_ctx(self):
        return gevent.Timeout(self.cfg.keepalive, False)

This logic will throw out gevent.Timeout type when timeout and our logic cannot handle that in all different kinds of exception logic until getting to the final branch: https://github.com/benoitc/gunicorn/blob/master/gunicorn/workers/base_async.py#L87
Because:

class Timeout(BaseException):

Impact
This will be a problem as a connection between the gateway (eg: Nginx) and Gunicorn will be suddenly closed without response and Nginx will expect the service was died and return back clients with error 502.

From Nginx you will see something that is not reasonable like:

2023/01/20 13:46:02 [error] 1027#1027: *209 upstream prematurely closed connection while reading response header from upstream, client: 127.0.0.1, server: flask_server.com, request: "GET /sleep?time=1 HTTP/1.1", upstream: "http://127.0.0.1:8000/sleep?time=1", host: "flask_server.com"

The error can also be triggered when throwing out any other exception that is not inherited from Exception but BaseException from user logic in Flask app.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions