Skip to content

gh-84559: Remove the new multiprocessing warning, too disruptive. #101551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions Doc/library/concurrent.futures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -281,18 +281,18 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.

Added the *initializer* and *initargs* arguments.

.. note::
The default :mod:`multiprocessing` start method
(see :ref:`multiprocessing-start-methods`) will change away from
*fork* in Python 3.14. Code that requires *fork* be used for their
:class:`ProcessPoolExecutor` should explicitly specify that by
passing a ``mp_context=multiprocessing.get_context("fork")``
parameter.

.. versionchanged:: 3.11
The *max_tasks_per_child* argument was added to allow users to
control the lifetime of workers in the pool.

.. versionchanged:: 3.12
The implicit use of the :mod:`multiprocessing` *fork* start method as a
platform default (see :ref:`multiprocessing-start-methods`) now raises a
:exc:`DeprecationWarning`. The default will change in Python 3.14.
Code that requires *fork* should explicitly specify that when creating
their :class:`ProcessPoolExecutor` by passing a
``mp_context=multiprocessing.get_context('fork')`` parameter.

.. _processpoolexecutor-example:

ProcessPoolExecutor Example
Expand Down
11 changes: 6 additions & 5 deletions Doc/library/multiprocessing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,11 @@ to start a process. These *start methods* are

Available on POSIX systems. Currently the default on POSIX except macOS.

.. note::
The default start method will change away from *fork* in Python 3.14.
Code that requires *fork* should explicitly specify that via
:func:`get_context` or :func:`set_start_method`.

*forkserver*
When the program starts and selects the *forkserver* start method,
a server process is spawned. From then on, whenever a new process
Expand All @@ -138,11 +143,6 @@ to start a process. These *start methods* are
Available on POSIX platforms which support passing file descriptors
over Unix pipes such as Linux.

.. versionchanged:: 3.12
Implicit use of the *fork* start method as the default now raises a
:exc:`DeprecationWarning`. Code that requires it should explicitly
specify *fork* via :func:`get_context` or :func:`set_start_method`.
The default will change away from *fork* in 3.14.

.. versionchanged:: 3.8

Expand Down Expand Up @@ -1107,6 +1107,7 @@ Miscellaneous
launched (before creating a :class:`Pool` or starting a :class:`Process`).

Only meaningful when using the ``'forkserver'`` start method.
See :ref:`multiprocessing-start-methods`.

.. versionadded:: 3.4

Expand Down
16 changes: 7 additions & 9 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -440,12 +440,6 @@ Deprecated
warning at compile time. This field will be removed in Python 3.14.
(Contributed by Ramvikrams and Kumar Aditya in :gh:`101193`. PEP by Ken Jin.)

* Use of the implicit default ``'fork'`` start method for
:mod:`multiprocessing` and :class:`concurrent.futures.ProcessPoolExecutor`
now emits a :exc:`DeprecationWarning` on Linux and other non-macOS POSIX
systems. Avoid this by explicitly specifying a start method.
See :ref:`multiprocessing-start-methods`.

Pending Removal in Python 3.13
------------------------------

Expand Down Expand Up @@ -510,9 +504,13 @@ Pending Removal in Python 3.14
* Testing the truth value of an :class:`xml.etree.ElementTree.Element`
is deprecated and will raise an exception in Python 3.14.

* The default :mod:`multiprocessing` start method will change to one of either
``'forkserver'`` or ``'spawn'`` on all platforms for which ``'fork'`` remains
the default per :gh:`84559`.
* The default :mod:`multiprocessing` start method will change to a safer one on
Linux, BSDs, and other non-macOS POSIX platforms where ``'fork'`` is currently
the default (:gh:`84559`). Adding a runtime warning about this was deemed too
disruptive as the majority of code is not expected to care. Use the
:func:`~multiprocessing.get_context` or
:func:`~multiprocessing.set_start_method` APIs to explicitly specify when
your code *requires* ``'fork'``. See :ref:`multiprocessing-start-methods`.

Pending Removal in Future Versions
----------------------------------
Expand Down
17 changes: 0 additions & 17 deletions Lib/concurrent/futures/process.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@
import itertools
import sys
from traceback import format_exception
import warnings


_threads_wakeups = weakref.WeakKeyDictionary()
Expand Down Expand Up @@ -651,22 +650,6 @@ def __init__(self, max_workers=None, mp_context=None,
mp_context = mp.get_context("spawn")
else:
mp_context = mp.get_context()
if (mp_context.get_start_method() == "fork" and
mp_context == mp.context._default_context._default_context):
warnings.warn(
"The default multiprocessing start method will change "
"away from 'fork' in Python >= 3.14, per GH-84559. "
"ProcessPoolExecutor uses multiprocessing. "
"If your application requires the 'fork' multiprocessing "
"start method, explicitly specify that by passing a "
"mp_context= parameter. "
"The safest start method is 'spawn'.",
category=mp.context.DefaultForkDeprecationWarning,
stacklevel=2,
)
# Avoid the equivalent warning from multiprocessing itself via
# a non-default fork context.
mp_context = mp.get_context("fork")
self._mp_context = mp_context

# https://github.com/python/cpython/issues/90622
Expand Down
28 changes: 1 addition & 27 deletions Lib/multiprocessing/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@ class TimeoutError(ProcessError):
class AuthenticationError(ProcessError):
pass

class DefaultForkDeprecationWarning(DeprecationWarning):
pass

#
# Base type for contexts. Bound methods of an instance of this type are included in __all__ of __init__.py
#
Expand Down Expand Up @@ -284,23 +281,6 @@ def _Popen(process_obj):
from .popen_fork import Popen
return Popen(process_obj)

_warn_package_prefixes = (os.path.dirname(__file__),)

class _DeprecatedForkProcess(ForkProcess):
@classmethod
def _Popen(cls, process_obj):
import warnings
warnings.warn(
"The default multiprocessing start method will change "
"away from 'fork' in Python >= 3.14, per GH-84559. "
"Use multiprocessing.get_context(X) or .set_start_method(X) to "
"explicitly specify it when your application requires 'fork'. "
"The safest start method is 'spawn'.",
category=DefaultForkDeprecationWarning,
skip_file_prefixes=_warn_package_prefixes,
)
return super()._Popen(process_obj)

class SpawnProcess(process.BaseProcess):
_start_method = 'spawn'
@staticmethod
Expand All @@ -324,9 +304,6 @@ class ForkContext(BaseContext):
_name = 'fork'
Process = ForkProcess

class _DefaultForkContext(ForkContext):
Process = _DeprecatedForkProcess

class SpawnContext(BaseContext):
_name = 'spawn'
Process = SpawnProcess
Expand All @@ -342,16 +319,13 @@ def _check_available(self):
'fork': ForkContext(),
'spawn': SpawnContext(),
'forkserver': ForkServerContext(),
# Remove None and _DefaultForkContext() when changing the default
# in 3.14 for https://github.com/python/cpython/issues/84559.
None: _DefaultForkContext(),
}
if sys.platform == 'darwin':
# bpo-33725: running arbitrary code after fork() is no longer reliable
# on macOS since macOS 10.14 (Mojave). Use spawn by default instead.
_default_context = DefaultContext(_concrete_contexts['spawn'])
else:
_default_context = DefaultContext(_concrete_contexts[None])
_default_context = DefaultContext(_concrete_contexts['fork'])

else:

Expand Down
5 changes: 2 additions & 3 deletions Lib/test/_test_multiprocessing.py
Original file line number Diff line number Diff line change
Expand Up @@ -4098,10 +4098,9 @@ def test_shared_memory_SharedMemoryServer_ignores_sigint(self):
def test_shared_memory_SharedMemoryManager_reuses_resource_tracker(self):
# bpo-36867: test that a SharedMemoryManager uses the
# same resource_tracker process as its parent.
cmd = f'''if 1:
cmd = '''if 1:
from multiprocessing.managers import SharedMemoryManager
from multiprocessing import set_start_method
set_start_method({multiprocessing.get_start_method()!r})


smm = SharedMemoryManager()
smm.start()
Expand Down
19 changes: 0 additions & 19 deletions Lib/test/test_concurrent_futures.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
import threading
import time
import unittest
import warnings
import weakref
from pickle import PicklingError

Expand Down Expand Up @@ -572,24 +571,6 @@ def test_shutdown_no_wait(self):
assert all([r == abs(v) for r, v in zip(res, range(-5, 5))])


@unittest.skipIf(mp.get_all_start_methods()[0] != "fork", "non-fork default.")
class ProcessPoolExecutorDefaultForkWarning(unittest.TestCase):
def test_fork_default_warns(self):
with self.assertWarns(mp.context.DefaultForkDeprecationWarning):
with futures.ProcessPoolExecutor(2):
pass

def test_explicit_fork_does_not_warn(self):
with warnings.catch_warnings(record=True) as ws:
warnings.simplefilter("ignore")
warnings.filterwarnings(
'always', category=mp.context.DefaultForkDeprecationWarning)
ctx = mp.get_context("fork") # Non-default fork context.
with futures.ProcessPoolExecutor(2, mp_context=ctx):
pass
self.assertEqual(len(ws), 0, msg=[str(x) for x in ws])


create_executor_tests(ProcessPoolShutdownTest,
executor_mixins=(ProcessPoolForkMixin,
ProcessPoolForkserverMixin,
Expand Down
5 changes: 2 additions & 3 deletions Lib/test/test_logging.py
Original file line number Diff line number Diff line change
Expand Up @@ -4759,9 +4759,8 @@ def test_multiprocessing(self):
# In other processes, processName is correct when multiprocessing in imported,
# but it is (incorrectly) defaulted to 'MainProcess' otherwise (bpo-38762).
import multiprocessing
mp = multiprocessing.get_context('spawn')
parent_conn, child_conn = mp.Pipe()
p = mp.Process(
parent_conn, child_conn = multiprocessing.Pipe()
p = multiprocessing.Process(
target=self._extract_logrecord_process_name,
args=(2, LOG_MULTI_PROCESSING, child_conn,)
)
Expand Down
85 changes: 0 additions & 85 deletions Lib/test/test_multiprocessing_defaults.py

This file was deleted.

3 changes: 1 addition & 2 deletions Lib/test/test_re.py
Original file line number Diff line number Diff line change
Expand Up @@ -2431,8 +2431,7 @@ def test_regression_gh94675(self):
input_js = '''a(function() {
///////////////////////////////////////////////////////////////////
});'''
mp = multiprocessing.get_context('spawn')
p = mp.Process(target=pattern.sub, args=('', input_js))
p = multiprocessing.Process(target=pattern.sub, args=('', input_js))
p.start()
p.join(SHORT_TIMEOUT)
try:
Expand Down

This file was deleted.