ssl module with OpenSSL 3.0 does not throw SSLEOFError on unclean shutdown #95494

davidben · 2022-07-31T05:37:52Z

Patch incoming. Filing this so there's a bug to attach the PR to.

GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify.

Linked PRs

pythonGH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?)

GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?)

…5495) pythonGH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?) (cherry picked from commit 420bbb7) Co-authored-by: David Benjamin <[email protected]>

…5495) pythonGH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?)

…#103006) GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?) (cherry picked from commit 420bbb7) Co-authored-by: David Benjamin <[email protected]>

ambv · 2023-03-27T14:15:19Z

This is now landed in all Python versions that support OpenSSL 3.0.

…#103007) GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?) (cherry picked from commit 420bbb7) Co-authored-by: David Benjamin <[email protected]>

…5495) pythonGH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right. That option causes OpenSSL to treat transport EOF as the same as close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl module would raise them for transport EOF and close_notify, respectively. In OpenSSL 3.0, both act like close_notify. Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING and mapping that to the other exception type. There doesn't seem to have been any unit test of this error, so fill in the missing one. This had to be done with the BIO path because it's actually slightly tricky to simulate a transport EOF with Python's fd based APIs. (If you instruct the server to close the socket, it gets confused, probably because the server's SSL object is still referencing the now dead fd?)

davidben added the type-bug An unexpected behavior, bug, or error label Jul 31, 2022

bedevere-bot mentioned this issue Jul 31, 2022

gh-95494: Fix transport EOF handling in OpenSSL 3.0 #95495

Merged

AlexWaygood added the topic-SSL label Aug 1, 2022

tiran added 3.11 only security fixes 3.10 only security fixes 3.12 only security fixes labels Aug 3, 2022

This was referenced Nov 3, 2022

SSLObject does not raise SSLEOFError on OpenSSL 3 #90471

Closed

Use OpenSSL 3.0.x in our binary builds #99079

Closed

AlexWaygood added this to OpenSSL 3 Nov 3, 2022

This was referenced Mar 24, 2023

[3.11] GH-95494: Fix transport EOF handling in OpenSSL 3.0 (GH-95495) #103006

Merged

[3.10] GH-95494: Fix transport EOF handling in OpenSSL 3.0 (GH-95495) #103007

Merged

ambv closed this as completed Mar 27, 2023

github-project-automation bot moved this to Done in OpenSSL 3 Mar 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ssl module with OpenSSL 3.0 does not throw SSLEOFError on unclean shutdown #95494

ssl module with OpenSSL 3.0 does not throw SSLEOFError on unclean shutdown #95494

davidben commented Jul 31, 2022 •

edited by bedevere-bot

Loading

ambv commented Mar 27, 2023

ssl module with OpenSSL 3.0 does not throw SSLEOFError on unclean shutdown #95494

ssl module with OpenSSL 3.0 does not throw SSLEOFError on unclean shutdown #95494

Comments

davidben commented Jul 31, 2022 • edited by bedevere-bot Loading

Linked PRs

ambv commented Mar 27, 2023

davidben commented Jul 31, 2022 •

edited by bedevere-bot

Loading