Skip to content

Segfault for short-lived Client objects in python 3.12 #1916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
basepi opened this issue Oct 9, 2023 · 2 comments
Closed

Segfault for short-lived Client objects in python 3.12 #1916

basepi opened this issue Oct 9, 2023 · 2 comments

Comments

@basepi
Copy link
Contributor

basepi commented Oct 9, 2023

This is a weird one.

If you have a short-lived Client object with the transport thread intact, it can cause a segfault.

This is what I've found after a ton of debugging:

  • It only happens in python 3.12, and only in CI (race condition?)
  • It appears to happen in the transport thread. You can see this if you remove the kwargs passed into test_python_version_deprecation (introduced in CI fixup #1913)
  • If you disable the metrics thread (with metrics_interval="0ms"), the segfault goes away
    • This is likely a red herring. I "fixed" the segfault here as well, and sending_elasticapm_client has the metrics thread disabled. But we do spin the threads up and down in a specific order, so perhaps the metrics thread is just changing the timing enough to cause the segfault in some cases?
  • If you disable the transport thread (with transport_class="tests.fixtures.DummyTransport), the segfault goes away
  • You can also see this failure in the test_wrapper_script_instrumentation test which is currently skipped in python 3.12.
    • You can stop the segfault with a five second sleep in testapp.py (thus the "short-lived Client object")
    • You can also stop the segfault by disabling either thread as mentioned above

With all of the above, I decided it wasn't a blocker, so I'm opening this technical debt issue. I don't think customers will run into this, but it is possible.

@basepi basepi added bug technical debt upstream Issue caused by an upstream package 8.12-candidate labels Oct 9, 2023
This was referenced Oct 9, 2023
@basepi
Copy link
Contributor Author

basepi commented Oct 10, 2023

For the record, I think this is a bug in Python 3.12. But I don't have enough information yet to open a helpful bug report.

@basepi
Copy link
Contributor Author

basepi commented Jan 18, 2024

Apparently, I forgot to update this thread! I was eventually able to find a reproduction case for this issue and opened this: python/cpython#111049

The issue was fixed in python/cpython#111221 and will be present in cpython 3.12.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant