Skip to content

Commit dc7c497

Browse files
committed
Merge remote-tracking branch 'upstream/main' into vfazio-thread_get_ident
2 parents 7bdfd34 + 25a7ddf commit dc7c497

File tree

7 files changed

+166
-41
lines changed

7 files changed

+166
-41
lines changed

Doc/library/configparser.rst

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1244,6 +1244,10 @@ ConfigParser Objects
12441244
*space_around_delimiters* is true, delimiters between
12451245
keys and values are surrounded by spaces.
12461246

1247+
.. versionchanged:: 3.14
1248+
Raises InvalidWriteError if this would write a representation which cannot
1249+
be accurately parsed by a future :meth:`read` call from this parser.
1250+
12471251
.. note::
12481252

12491253
Comments in the original configuration file are not preserved when
@@ -1459,6 +1463,17 @@ Exceptions
14591463

14601464
.. versionadded:: 3.14
14611465

1466+
.. exception:: InvalidWriteError
1467+
1468+
Exception raised when an attempted :meth:`ConfigParser.write` would not be parsed
1469+
accurately with a future :meth:`ConfigParser.read` call.
1470+
1471+
Ex: Writing a key beginning with the :attr:`ConfigParser.SECTCRE` pattern
1472+
would parse as a section header when read. Attempting to write this will raise
1473+
this exception.
1474+
1475+
.. versionadded:: 3.14
1476+
14621477
.. rubric:: Footnotes
14631478

14641479
.. [1] Config parsers allow for heavy customization. If you are interested in

Doc/library/multiprocessing.rst

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,8 @@ Contexts and start methods
107107
Depending on the platform, :mod:`multiprocessing` supports three ways
108108
to start a process. These *start methods* are
109109

110+
.. _multiprocessing-start-method-spawn:
111+
110112
*spawn*
111113
The parent process starts a fresh Python interpreter process. The
112114
child process will only inherit those resources necessary to run
@@ -117,6 +119,8 @@ to start a process. These *start methods* are
117119

118120
Available on POSIX and Windows platforms. The default on Windows and macOS.
119121

122+
.. _multiprocessing-start-method-fork:
123+
120124
*fork*
121125
The parent process uses :func:`os.fork` to fork the Python
122126
interpreter. The child process, when it begins, is effectively
@@ -137,6 +141,8 @@ to start a process. These *start methods* are
137141
raise a :exc:`DeprecationWarning`. Use a different start method.
138142
See the :func:`os.fork` documentation for further explanation.
139143

144+
.. _multiprocessing-start-method-forkserver:
145+
140146
*forkserver*
141147
When the program starts and selects the *forkserver* start method,
142148
a server process is spawned. From then on, whenever a new process
@@ -2987,6 +2993,9 @@ Beware of replacing :data:`sys.stdin` with a "file like object"
29872993

29882994
For more information, see :issue:`5155`, :issue:`5313` and :issue:`5331`
29892995

2996+
.. _multiprocessing-programming-spawn:
2997+
.. _multiprocessing-programming-forkserver:
2998+
29902999
The *spawn* and *forkserver* start methods
29913000
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
29923001

Doc/whatsnew/3.14.rst

Lines changed: 54 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,22 @@ Summary -- release highlights
7070
* :ref:`A new type of interpreter <whatsnew314-tail-call>`
7171

7272

73+
Incompatible changes
74+
====================
75+
76+
On platforms other than macOS and Windows, the default :ref:`start
77+
method <multiprocessing-start-methods>` for :mod:`multiprocessing`
78+
and :class:`~concurrent.futures.ProcessPoolExecutor` switches from
79+
*fork* to *forkserver*.
80+
81+
See :ref:`(1) <whatsnew314-concurrent-futures-start-method>` and
82+
:ref:`(2) <whatsnew314-multiprocessing-start-method>` for details.
83+
84+
If you encounter :exc:`NameError`\s or pickling errors coming out of
85+
:mod:`multiprocessing` or :mod:`concurrent.futures`, see the
86+
:ref:`forkserver restrictions <multiprocessing-programming-forkserver>`.
87+
88+
7389
New features
7490
============
7591

@@ -396,12 +412,26 @@ concurrent.futures
396412
same process) to Python code. This is separate from the proposed API
397413
in :pep:`734`.
398414
(Contributed by Eric Snow in :gh:`124548`.)
399-
* The default ``ProcessPoolExecutor`` start method (see
400-
:ref:`multiprocessing-start-methods`) changed from *fork* to *forkserver* on
401-
platforms other than macOS & Windows. If you require the threading
402-
incompatible *fork* start method you must explicitly request it by
403-
supplying a *mp_context* to :class:`concurrent.futures.ProcessPoolExecutor`.
404-
(Contributed by Gregory P. Smith in :gh:`84559`.)
415+
416+
.. _whatsnew314-concurrent-futures-start-method:
417+
418+
* The default :class:`~concurrent.futures.ProcessPoolExecutor`
419+
:ref:`start method <multiprocessing-start-methods>` changed
420+
from :ref:`fork <multiprocessing-start-method-fork>` to :ref:`forkserver
421+
<multiprocessing-start-method-forkserver>` on platforms other than macOS and
422+
Windows where it was already :ref:`spawn <multiprocessing-start-method-spawn>`.
423+
424+
If the threading incompatible *fork* method is required, you must explicitly
425+
request it by supplying a multiprocessing context *mp_context* to
426+
:class:`~concurrent.futures.ProcessPoolExecutor`.
427+
428+
See :ref:`forkserver restrictions <multiprocessing-programming-forkserver>`
429+
for information and differences with the *fork* method and how this change
430+
may affect existing code with mutable global shared variables and/or shared
431+
objects that can not be automatically :mod:`pickled <pickle>`.
432+
433+
(Contributed by Gregory P. Smith in :gh:`84559`.)
434+
405435

406436
contextvars
407437
-----------
@@ -637,18 +667,30 @@ mimetypes
637667
multiprocessing
638668
---------------
639669

640-
* The default start method (see :ref:`multiprocessing-start-methods`) changed
641-
from *fork* to *forkserver* on platforms other than macOS & Windows where
642-
it was already *spawn*. If you require the threading incompatible *fork*
643-
start method you must explicitly request it using a context from
644-
:func:`multiprocessing.get_context` (preferred) or change the default via
645-
:func:`multiprocessing.set_start_method`.
670+
.. _whatsnew314-multiprocessing-start-method:
671+
672+
* The default :ref:`start method <multiprocessing-start-methods>` changed
673+
from :ref:`fork <multiprocessing-start-method-fork>` to :ref:`forkserver
674+
<multiprocessing-start-method-forkserver>` on platforms other than macOS and
675+
Windows where it was already :ref:`spawn <multiprocessing-start-method-spawn>`.
676+
677+
If the threading incompatible *fork* method is required, you must explicitly
678+
request it via a context from :func:`multiprocessing.get_context` (preferred)
679+
or change the default via :func:`multiprocessing.set_start_method`.
680+
681+
See :ref:`forkserver restrictions <multiprocessing-programming-forkserver>`
682+
for information and differences with the *fork* method and how this change
683+
may affect existing code with mutable global shared variables and/or shared
684+
objects that can not be automatically :mod:`pickled <pickle>`.
685+
646686
(Contributed by Gregory P. Smith in :gh:`84559`.)
687+
647688
* :mod:`multiprocessing`'s ``"forkserver"`` start method now authenticates
648689
its control socket to avoid solely relying on filesystem permissions
649690
to restrict what other processes could cause the forkserver to spawn workers
650691
and run code.
651692
(Contributed by Gregory P. Smith for :gh:`97514`.)
693+
652694
* The :ref:`multiprocessing proxy objects <multiprocessing-proxy_objects>`
653695
for *list* and *dict* types gain previously overlooked missing methods:
654696

Lib/configparser.py

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@
161161
"InterpolationMissingOptionError", "InterpolationSyntaxError",
162162
"ParsingError", "MissingSectionHeaderError",
163163
"MultilineContinuationError", "UnnamedSectionDisabledError",
164-
"ConfigParser", "RawConfigParser",
164+
"InvalidWriteError", "ConfigParser", "RawConfigParser",
165165
"Interpolation", "BasicInterpolation", "ExtendedInterpolation",
166166
"SectionProxy", "ConverterMapping",
167167
"DEFAULTSECT", "MAX_INTERPOLATION_DEPTH", "UNNAMED_SECTION")
@@ -375,6 +375,14 @@ class _UnnamedSection:
375375
def __repr__(self):
376376
return "<UNNAMED_SECTION>"
377377

378+
class InvalidWriteError(Error):
379+
"""Raised when attempting to write data that the parser would read back differently.
380+
ex: writing a key which begins with the section header pattern would read back as a
381+
new section """
382+
383+
def __init__(self, msg=''):
384+
Error.__init__(self, msg)
385+
378386

379387
UNNAMED_SECTION = _UnnamedSection()
380388

@@ -973,6 +981,7 @@ def _write_section(self, fp, section_name, section_items, delimiter, unnamed=Fal
973981
if not unnamed:
974982
fp.write("[{}]\n".format(section_name))
975983
for key, value in section_items:
984+
self._validate_key_contents(key)
976985
value = self._interpolation.before_write(self, section_name, key,
977986
value)
978987
if value is not None or not self._allow_no_value:
@@ -1210,6 +1219,14 @@ def _convert_to_boolean(self, value):
12101219
raise ValueError('Not a boolean: %s' % value)
12111220
return self.BOOLEAN_STATES[value.lower()]
12121221

1222+
def _validate_key_contents(self, key):
1223+
"""Raises an InvalidWriteError for any keys containing
1224+
delimiters or that match the section header pattern"""
1225+
if re.match(self.SECTCRE, key):
1226+
raise InvalidWriteError("Cannot write keys matching section pattern")
1227+
if any(delim in key for delim in self._delimiters):
1228+
raise InvalidWriteError("Cannot write key that contains delimiters")
1229+
12131230
def _validate_value_types(self, *, section="", option="", value=""):
12141231
"""Raises a TypeError for illegal non-string values.
12151232

Lib/test/test_configparser.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2192,6 +2192,30 @@ def test_multiple_configs(self):
21922192
self.assertEqual('2', cfg[configparser.UNNAMED_SECTION]['b'])
21932193

21942194

2195+
class InvalidInputTestCase(unittest.TestCase):
2196+
"""Tests for issue #65697, where configparser will write configs
2197+
it parses back differently. Ex: keys containing delimiters or
2198+
matching the section pattern"""
2199+
2200+
def test_delimiter_in_key(self):
2201+
cfg = configparser.ConfigParser(delimiters=('='))
2202+
cfg.add_section('section1')
2203+
cfg.set('section1', 'a=b', 'c')
2204+
output = io.StringIO()
2205+
with self.assertRaises(configparser.InvalidWriteError):
2206+
cfg.write(output)
2207+
output.close()
2208+
2209+
def test_section_bracket_in_key(self):
2210+
cfg = configparser.ConfigParser()
2211+
cfg.add_section('section1')
2212+
cfg.set('section1', '[this parses back as a section]', 'foo')
2213+
output = io.StringIO()
2214+
with self.assertRaises(configparser.InvalidWriteError):
2215+
cfg.write(output)
2216+
output.close()
2217+
2218+
21952219
class MiscTestCase(unittest.TestCase):
21962220
def test__all__(self):
21972221
support.check__all__(self, configparser, not_exported={"Error"})
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
stdlib configparser will now attempt to validate that keys it writes will not result in file corruption (creating a file unable to be accurately parsed by a future read() call from the same parser). Attempting a corrupting write() will raise an InvalidWriteError.

0 commit comments

Comments
 (0)