-
-
Notifications
You must be signed in to change notification settings - Fork 5.4k
BUG: _lib: crash in uarray on interpreter exit with a free-threaded interpreter #21214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Labels
defect
A clear bug or issue that prevents SciPy from being installed or used as expected
free-threading
Items related to supporting free-threaded (a.k.a. "no-GIL") builds of CPython
scipy._lib
uarray
Items related to the uarray backend
Milestone
Comments
51 tasks
tylerjereddy
pushed a commit
to tylerjereddy/scipy
that referenced
this issue
Feb 5, 2025
…efcounts (scipy#22038) #### Reference issue Fixes scipy#21214 Closes scipy#21218 #### What does this implement/fix? This PR addresses the segfaults caused by the occurrence of negative refcounts when uarray static strings were being released when the interpreter didn't exist. --------- Co-authored-by: peterbell10 <[email protected]>
tylerjereddy
added a commit
that referenced
this issue
Feb 16, 2025
* BUG: immortalize uarray global strings in order to prevent negative refcounts (#22038) #### Reference issue Fixes #21214 Closes #21218 #### What does this implement/fix? This PR addresses the segfaults caused by the occurrence of negative refcounts when uarray static strings were being released when the interpreter didn't exist. --------- Co-authored-by: peterbell10 <[email protected]> * BUG: special: Fix unchecked memory allocations in `specfun.h` (#22080) * WIP: Fix unchecked memory allocation in aswfa() Summary of the changes: * Add another SF error code, "SF_ERROR_NOMEM", that is used by ufuncs that require internally allocated memory. Just like the other possible errors, how this error is handled can be queried and set using `geterr()` and `seterr()`. Unlike the other error conditions, the default behavior is to raise an exception if a memory allocation fails. * Modify the function `aswfa()` in `specfun.h` to return a status code that indicates when a mempory allocation failed. * Modify the functions `prolate_aswfa_nocv()`, `oblate_aswfa_nocv()`, `prolate_aswfa()` and `oblate_aswfa()` in `sphd_wave.h` to check the return value of `aswfa()`, and set the error state and return values appropriately if `aswfa()` returns `NoMemory`. * MAINT: Rename 'nomem' -> 'memory'. * Fix all uses of unchecked allocations in specfun.h * Fix aswfa() in specfunc.h * Fix refguide check. * Add comment about the return value when nonconvergence is detected in rmn2l. * Change (c/m)alloc calls to use 'new' wrapped in a unique_ptr. * Fix a few more names passed to set_error() * Set outputs to nan in mtu12 when alloc fails Also update docstring-like comments of some C++ functions. * BUG: cluster: `cophenet` intercept invalid linkage matrix count (#22187) * BUG: catch invalid linkage count * Fixes gh-22183. * The input data from gh-22183 causes an out of bounds memory access in a 1-D `memoryview` in the Cython `cophenetic_distances` function. Prevent this by enforcing a check for an allowable upper bound on the cluster membership (4th column) of the linkage matrix `Z` received by `is_valid_linkage`. Co-authored-by: Warren Weckesser <[email protected]> * TST: PR 22187 revisions * Adjust `test_gh_22183()` such that the linkage matrix is invalid only because of an excessive observation count, rather than also being invalid because the large observation count is not integral. * Adjust `test_gh_22183()` such that is properly flushes through array API backends. * DOC: backticks --------- Co-authored-by: Warren Weckesser <[email protected]> Co-authored-by: Jake Bowhay <[email protected]> * MAINT: stats.Mixture: make return type consistent * MAINT: stats.Mixture: fix inverse functions when mean is undefined (#22337) * MAINT: stats.Mixture: fix inverse when mean is undefined * Apply suggestions from code review * Update scipy/stats/_distribution_infrastructure.py * BUG: special: Fix unchecked malloc in stirling2.h (#22339) Use std::unique_ptr and new (std::nothrow) instead of malloc. Closes gh-22336. * BUG: sparse: fix selecting wrong dtype for coo coords (#22353) * Fixes bug with selecting wrong dtype for coo coords * add test for reshape having wrong dtype * test that smaller dtype is maintained even if intermediate values are big --------- Co-authored-by: Dan Schult <[email protected]> * TST: interpolate: small tolerance bump to TestAAA.test_basic_functions * MAINT: Stop installing libhighs Co-authored-by: drew-parsons <[email protected]> Closes #22349 * BUG: sparse.linalg.norm: fix return type (#22372) cupy reports cupy/cupy#8866 that sparse.linalg.norm returns a python number instead of a 0-dim numpy array * revert NotImplemented return values in dot & others (#22373) * DOC: sparse.linalg: add two recent functions to namespace and fix doctests (#22374) * fix doctests and make two recent functions visible * update refguide [docs only] * fix capitalization [docs only] Co-authored-by: Daniel Schmitz <[email protected]> --------- Co-authored-by: Daniel Schmitz <[email protected]> * MAINT: stats.zmap: restore support for complex data (#22405) * MAINT: stats.zmap: restore support for complex data * MAINT: stats.zmap: fix masked array support * Apply suggestions from code review * BUG: scipy.spatial: Fix inaccurate orthonormalization in `Rotation.from_matrix()` (#22418) * BUG: fix inaccurate orthonormalization in Rotation.from_matrix() * Code review updates --------- Co-authored-by: Scott Shambaugh <[email protected]> * BUG: special: Fix a memory leak in the AMOS function besy(). (#22423) Use a unique_ptr to manage the heap-allocated array. Closes gh-22314. * CI: build Linux aarch64 wheels on GitHub Actions instead of Cirrus CI This unifies the wheels builds, which is a lot easier on maintenance. In the process, did the following: - update to setup-miniconda 3.1.1. for GHA Linux arm64 runner support - add `conda-remove-defaults` to `setup-miniconda` * CI: migrate Linux aarch64 job to GitHub Actions * CI: remove Cirrus CI configuration [wheel build] * BUG: io.loadmat: throw error on file containing all zeros (#22447) [ci skip] * BUG: PR 22471 revisions * Fixes issues related to backport manual merge conflict resolution during the preparation of SciPy `1.15.2`. * Restore an accidentally-removed size-0 array check in `_contains_nan()` function--this allows SciPy `stats` tests to pass again on the maintenance branch. * Remove some accidentally-added tests in `test_mio.py` [skip circle] * BUG: sparse: fix spmatrix indexing with None and implicit padding to match matrix behavior (#22472) * fix index using None when index is padded with : * align None indexing for spmatrix with np.matrix behavior. And tests. * rewrite spmatrix handling to ease reading * MAINT: pearsonr SIMD-related shim * Fixes #22479. * This small patch allows `test_stats.py::TestRegression::test_regressZEROX` to pass on x86_64 Linux instead of failing due to an extra division by zero warning over a fairly narrow range of NumPy versions near `1.25.2` where SIMD implementation details appear to have been slightly different. [skip circle] * DOC: PR 22471 revisions * Update the SciPy `1.15.2` release notes following backport activity. * BUG: wrap median_filter stability (#22402) * An uninitialized data structure in `_rank_filter()` C++ code was causing stochastic algorithm failures on Linux but not MacOS, so force a zero-based initialization. This allows `test_gh_22250` to pass over thousands of `pytest-repeat` incantations. * Add a regression test for gh-22333, which reliably passes on this branch with `pytest-repeat` (and fails on `main` with sufficient repeats). * MAINT: integrate.cumulative_simpson: bump test tolerance * do not check dtype in test_compare_with_GCVSPL * CI: PR 22471 revisions * temporarily disable testing against `meson` `master` branch in CI because of gh-22534. * DOC: PR 22471 revisions * Update the SciPy `1.15.2` release notes following additional backport activity. * CI: PR 22471 wheel builds [wheel build] * This is an empty commit to test the wheel build matrix in PR 22471. [wheel build] --------- Co-authored-by: Edgar Andrés Margffoy Tuay <[email protected]> Co-authored-by: peterbell10 <[email protected]> Co-authored-by: Warren Weckesser <[email protected]> Co-authored-by: Jake Bowhay <[email protected]> Co-authored-by: Matt Haberland <[email protected]> Co-authored-by: Parth Nobel <[email protected]> Co-authored-by: Dan Schult <[email protected]> Co-authored-by: Rohit Goswami <[email protected]> Co-authored-by: Daniel Schmitz <[email protected]> Co-authored-by: Scott Shambaugh <[email protected]> Co-authored-by: Scott Shambaugh <[email protected]> Co-authored-by: Ralf Gommers <[email protected]> Co-authored-by: Matthew Brett <[email protected]> Co-authored-by: Charles Bousseau <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
defect
A clear bug or issue that prevents SciPy from being installed or used as expected
free-threading
Items related to supporting free-threaded (a.k.a. "no-GIL") builds of CPython
scipy._lib
uarray
Items related to the uarray backend
From @albanD's report in #20669 (comment):
Running with pydebug build as of two weeks ago and using ua_function, 1.15.0dev0 crashes on exit with
From @colesbury comment that 3.13t sets all module as being immortal, the explanation is that
scipy/scipy/_lib/_uarray/_uarray_dispatch.cxx
Line 1774 in 941d5b0
scipy/scipy/_lib/_uarray/_uarray_dispatch.cxx
Line 145 in 941d5b0
When we get to static variable de-alloc, we end up calling the py_ref destructor
scipy/scipy/_lib/_uarray/_uarray_dispatch.cxx
Line 38 in 941d5b0
Since the m_free not being called is not going to be fixed, I guess the right fix is to have the static variable have an owning reference to the py_obj but explicitly leak it (never call decref on it).
This would be the same treatment being applied to PyTorch's static variables in pytorch/pytorch#130341 and other places like https://github.com/pytorch/pytorch/blob/0b134c15cdf8cb4c2c7ffed1c2b6518de44382bf/torch/csrc/autograd/python_hook.cpp#L105
The text was updated successfully, but these errors were encountered: