Merge branch main into main-slp

Author: Anselm Kruis

Doc/howto/regex.rst

@@ -96,8 +96,9 @@ special nature.
 
 You can match the characters not listed within the class by :dfn:`complementing`
 the set.  This is indicated by including a ``'^'`` as the first character of the
-class; ``'^'`` outside a character class will simply match the ``'^'``
-character.  For example, ``[^5]`` will match any character except ``'5'``.
+class. For example, ``[^5]`` will match any character except ``'5'``.  If the
+caret appears elsewhere in a character class, it does not have special meaning.
+For example: ``[5^]`` will match either a ``'5'`` or a ``'^'``.
 
 Perhaps the most important metacharacter is the backslash, ``\``.   As in Python
 string literals, the backslash can be followed by various characters to signal

Doc/library/asyncio-eventloop.rst

@@ -1394,8 +1394,7 @@ Do not instantiate the class directly.
 
    .. attribute:: sockets
 
-      List of :class:`socket.socket` objects the server is listening on,
-      or ``None`` if the server is closed.
+      List of :class:`socket.socket` objects the server is listening on.
 
       .. versionchanged:: 3.7
          Prior to Python 3.7 ``Server.sockets`` used to return an

Doc/library/collections.rst

@@ -100,6 +100,21 @@ The class can be used to simulate nested scopes and is useful in templating.
         :func:`super` function.  A reference to ``d.parents`` is equivalent to:
         ``ChainMap(*d.maps[1:])``.
 
+    Note, the iteration order of a :class:`ChainMap()` is determined by
+    scanning the mappings last to first::
+
+        >>> baseline = {'music': 'bach', 'art': 'rembrandt'}
+        >>> adjustments = {'art': 'van gogh', 'opera': 'carmen'}
+        >>> list(ChainMap(adjustments, baseline))
+        ['music', 'art', 'opera']
+
+    This gives the same ordering as a series of :meth:`dict.update` calls
+    starting with the last mapping::
+
+        >>> combined = baseline.copy()
+        >>> combined.update(adjustments)
+        >>> list(combined)
+        ['music', 'art', 'opera']
 
 .. seealso::
 
@@ -163,8 +178,8 @@ contexts::
         e.maps[-1]            # Root context -- like Python's globals()
         e.parents             # Enclosing context chain -- like Python's nonlocals
 
-        d['x']                # Get first key in the chain of contexts
         d['x'] = 1            # Set value in current context
+        d['x']                # Get first key in the chain of contexts
         del d['x']            # Delete from current context
         list(d)               # All nested values
         k in d                # Check all nested values
@@ -253,15 +268,20 @@ For example::
 
     .. versionadded:: 3.1
 
+    .. versionchanged:: 3.7 As a :class:`dict` subclass, :class:`Counter`
+       Inherited the capability to remember insertion order.  Math operations
+       on *Counter* objects also preserve order.  Results are ordered
+       according to when an element is first encountered in the left operand
+       and then by the order encountered in the right operand.
 
     Counter objects support three methods beyond those available for all
     dictionaries:
 
     .. method:: elements()
 
         Return an iterator over elements repeating each as many times as its
-        count.  Elements are returned in arbitrary order.  If an element's count
-        is less than one, :meth:`elements` will ignore it.
+        count.  Elements are returned in the order first encountered. If an
+        element's count is less than one, :meth:`elements` will ignore it.
 
             >>> c = Counter(a=4, b=2, c=0, d=-2)
             >>> sorted(c.elements())
@@ -272,10 +292,10 @@ For example::
         Return a list of the *n* most common elements and their counts from the
         most common to the least.  If *n* is omitted or ``None``,
         :meth:`most_common` returns *all* elements in the counter.
-        Elements with equal counts are ordered arbitrarily:
+        Elements with equal counts are ordered in the order first encountered:
 
-            >>> Counter('abracadabra').most_common(3)  # doctest: +SKIP
-            [('a', 5), ('r', 2), ('b', 2)]
+            >>> Counter('abracadabra').most_common(3)
+            [('a', 5), ('b', 2), ('r', 2)]
 
     .. method:: subtract([iterable-or-mapping])
 
@@ -887,7 +907,7 @@ field names, the method and attribute names start with an underscore.
 
 .. method:: somenamedtuple._asdict()
 
-    Return a new :class:`OrderedDict` which maps field names to their corresponding
+    Return a new :class:`dict` which maps field names to their corresponding
     values:
 
     .. doctest::
@@ -1024,17 +1044,41 @@ customize a prototype instance:
 :class:`OrderedDict` objects
 ----------------------------
 
-Ordered dictionaries are just like regular dictionaries but they remember the
-order that items were inserted.  When iterating over an ordered dictionary,
-the items are returned in the order their keys were first added.
+Ordered dictionaries are just like regular dictionaries but have some extra
+capabilities relating to ordering operations.  They have become less
+important now that the built-in :class:`dict` class gained the ability
+to remember insertion order (this new behavior became guaranteed in
+Python 3.7).
+
+Some differences from :class:`dict` still remain:
+
+* The regular :class:`dict` was designed to be very good at mapping
+  operations.  Tracking insertion order was secondary.
+
+* The :class:`OrderedDict` was designed to be good at reordering operations.
+  Space efficiency, iteration speed, and the performance of update
+  operations were secondary.
+
+* Algorithmically, :class:`OrderedDict` can handle frequent reordering
+  operations better than :class:`dict`.  This makes it suitable for tracking
+  recent accesses (for example in an `LRU cache
+  <https://medium.com/@krishankantsinghal/my-first-blog-on-medium-583159139237>`_).
+
+* The equality operation for :class:`OrderedDict` checks for matching order.
+
+* The :meth:`popitem` method of :class:`OrderedDict` has a different
+  signature.  It accepts an optional argument to specify which item is popped.
+
+* :class:`OrderedDict` has a :meth:`move_to_end` method to
+  efficiently reposition an element to an endpoint.
+
+* Until Python 3.8, :class:`dict` lacked a :meth:`__reversed__` method.
+
 
 .. class:: OrderedDict([items])
 
-    Return an instance of a dict subclass, supporting the usual :class:`dict`
-    methods.  An *OrderedDict* is a dict that remembers the order that keys
-    were first inserted. If a new entry overwrites an existing entry, the
-    original insertion position is left unchanged.  Deleting an entry and
-    reinserting it will move it to the end.
+    Return an instance of a :class:`dict` subclass that has methods
+    specialized for rearranging dictionary order.
 
     .. versionadded:: 3.1
 
@@ -1084,29 +1128,7 @@ anywhere a regular dictionary is used.
 :class:`OrderedDict` Examples and Recipes
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Since an ordered dictionary remembers its insertion order, it can be used
-in conjunction with sorting to make a sorted dictionary::
-
-    >>> # regular unsorted dictionary
-    >>> d = {'banana': 3, 'apple': 4, 'pear': 1, 'orange': 2}
-
-    >>> # dictionary sorted by key
-    >>> OrderedDict(sorted(d.items(), key=lambda t: t[0]))
-    OrderedDict([('apple', 4), ('banana', 3), ('orange', 2), ('pear', 1)])
-
-    >>> # dictionary sorted by value
-    >>> OrderedDict(sorted(d.items(), key=lambda t: t[1]))
-    OrderedDict([('pear', 1), ('orange', 2), ('banana', 3), ('apple', 4)])
-
-    >>> # dictionary sorted by length of the key string
-    >>> OrderedDict(sorted(d.items(), key=lambda t: len(t[0])))
-    OrderedDict([('pear', 1), ('apple', 4), ('orange', 2), ('banana', 3)])
-
-The new sorted dictionaries maintain their sort order when entries
-are deleted.  But when new keys are added, the keys are appended
-to the end and the sort is not maintained.
-
-It is also straight-forward to create an ordered dictionary variant
+It is straightforward to create an ordered dictionary variant
 that remembers the order the keys were *last* inserted.
 If a new entry overwrites an existing entry, the
 original insertion position is changed and moved to the end::
@@ -1115,21 +1137,29 @@ original insertion position is changed and moved to the end::
         'Store items in the order the keys were last added'
 
         def __setitem__(self, key, value):
-            if key in self:
-                del self[key]
-            OrderedDict.__setitem__(self, key, value)
+            super().__setitem__(key, value)
+            super().move_to_end(key)
 
-An ordered dictionary can be combined with the :class:`Counter` class
-so that the counter remembers the order elements are first encountered::
+An :class:`OrderedDict` would also be useful for implementing
+variants of :func:`functools.lru_cache`::
 
-    class OrderedCounter(Counter, OrderedDict):
-        'Counter that remembers the order elements are first encountered'
+    class LRU(OrderedDict):
+        'Limit size, evicting the least recently looked-up key when full'
 
-        def __repr__(self):
-            return '%s(%r)' % (self.__class__.__name__, OrderedDict(self))
+        def __init__(self, maxsize=128, *args, **kwds):
+            self.maxsize = maxsize
+            super().__init__(*args, **kwds)
 
-        def __reduce__(self):
-            return self.__class__, (OrderedDict(self),)
+        def __getitem__(self, key):
+            value = super().__getitem__(key)
+            self.move_to_end(key)
+            return value
+
+        def __setitem__(self, key, value):
+            super().__setitem__(key, value)
+            if len(self) > self.maxsize:
+                oldest = next(iter(self))
+                del self[oldest]
 
 
 :class:`UserDict` objects

Doc/library/functions.rst

@@ -816,7 +816,7 @@ are always available.  They are listed here in alphabetical order.
 
       from functools import partial
       with open('mydata.db', 'rb') as f:
-          for block in iter(partial(f.read, 64), ''):
+          for block in iter(partial(f.read, 64), b''):
               process_block(block)
 
 

Doc/library/gc.rst

@@ -63,11 +63,14 @@ The :mod:`gc` module provides the following functions:
    Return the debugging flags currently set.
 
 
-.. function:: get_objects()
+.. function:: get_objects(generation=None)
 
    Returns a list of all objects tracked by the collector, excluding the list
-   returned.
+   returned. If *generation* is not None, return only the objects tracked by
+   the collector that are in that generation.
 
+   .. versionchanged:: 3.8
+      New *generation* parameter.
 
 .. function:: get_stats()
 

Doc/library/random.rst

@@ -404,7 +404,7 @@ with replacement to estimate a confidence interval for the mean of a sample of
 size five::
 
    # http://statistics.about.com/od/Applications/a/Example-Of-Bootstrapping.htm
-   from statistics import mean
+   from statistics import fmean as mean
    from random import choices
 
    data = 1, 2, 4, 4, 10
@@ -419,7 +419,7 @@ to determine the statistical significance or `p-value
 between the effects of a drug versus a placebo::
 
     # Example from "Statistics is Easy" by Dennis Shasha and Manda Wilson
-    from statistics import mean
+    from statistics import fmean as mean
     from random import shuffle
 
     drug = [54, 73, 53, 70, 73, 68, 52, 65, 65]

Doc/library/statistics.rst

@@ -39,6 +39,7 @@ or sample.
 
 =======================  =============================================
 :func:`mean`             Arithmetic mean ("average") of data.
+:func:`fmean`            Fast, floating point arithmetic mean.
 :func:`harmonic_mean`    Harmonic mean of data.
 :func:`median`           Median (middle value) of data.
 :func:`median_low`       Low median of data.
@@ -111,6 +112,23 @@ However, for reading convenience, most of the examples show sorted sequences.
       ``mean(data)`` is equivalent to calculating the true population mean μ.
 
 
+.. function:: fmean(data)
+
+   Convert *data* to floats and compute the arithmetic mean.
+
+   This runs faster than the :func:`mean` function and it always returns a
+   :class:`float`.  The result is highly accurate but not as perfect as
+   :func:`mean`.  If the input dataset is empty, raises a
+   :exc:`StatisticsError`.
+
+   .. doctest::
+
+      >>> fmean([3.5, 4.0, 5.25])
+      4.25
+
+   .. versionadded:: 3.8
+
+
 .. function:: harmonic_mean(data)
 
    Return the harmonic mean of *data*, a sequence or iterator of

Doc/library/test.rst

@@ -356,6 +356,10 @@ The :mod:`test.support` module defines the following constants:
 
    Check for presence of docstrings.
 
+.. data:: TEST_HTTP_URL
+
+   Define the URL of a dedicated HTTP server for the network tests.
+
 
 
 The :mod:`test.support` module defines the following functions:
@@ -936,9 +940,24 @@ The :mod:`test.support` module defines the following functions:
 
    Test for syntax errors in *statement* by attempting to compile *statement*.
    *testcase* is the :mod:`unittest` instance for the test.  *errtext* is the
-   text of the error raised by :exc:`SyntaxError`.  If *lineno* is not None,
-   compares to the line of the :exc:`SyntaxError`.  If *offset* is not None,
-   compares to the offset of the :exc:`SyntaxError`.
+   regular expression which should match the string representation of the
+   raised :exc:`SyntaxError`.  If *lineno* is not ``None``, compares to
+   the line of the exception.  If *offset* is not ``None``, compares to
+   the offset of the exception.
+
+
+.. function:: check_syntax_warning(testcase, statement, errtext='', *, lineno=1, offset=None)
+
+   Test for syntax warning in *statement* by attempting to compile *statement*.
+   Test also that the :exc:`SyntaxWarning` is emitted only once, and that it
+   will be converted to a :exc:`SyntaxError` when turned into error.
+   *testcase* is the :mod:`unittest` instance for the test.  *errtext* is the
+   regular expression which should match the string representation of the
+   emitted :exc:`SyntaxWarning` and raised :exc:`SyntaxError`.  If *lineno*
+   is not ``None``, compares to the line of the warning and exception.
+   If *offset* is not ``None``, compares to the offset of the exception.
+
+   .. versionadded:: 3.8
 
 
 .. function:: open_urlresource(url, *args, **kw)

Doc/library/unittest.rst

@@ -27,7 +27,7 @@ object-oriented way:
 
 test fixture
    A :dfn:`test fixture` represents the preparation needed to perform one or more
-   tests, and any associate cleanup actions.  This may involve, for example,
+   tests, and any associated cleanup actions.  This may involve, for example,
    creating temporary or proxy databases, directories, or starting a server
    process.
 
@@ -2083,7 +2083,7 @@ Loading and running tests
 
    .. method:: run(test)
 
-      This method is the main public interface to the `TextTestRunner`. This
+      This method is the main public interface to the ``TextTestRunner``. This
       method takes a :class:`TestSuite` or :class:`TestCase` instance. A
       :class:`TestResult` is created by calling
       :func:`_makeResult` and the test(s) are run and the

Doc/library/xml.etree.elementtree.rst

@@ -494,10 +494,12 @@ Functions
       by the user.
 
 
-.. function:: fromstring(text)
+.. function:: fromstring(text, parser=None)
 
    Parses an XML section from a string constant.  Same as :func:`XML`.  *text*
-   is a string containing XML data.  Returns an :class:`Element` instance.
+   is a string containing XML data.  *parser* is an optional parser instance.
+   If not given, the standard :class:`XMLParser` parser is used.
+   Returns an :class:`Element` instance.
 
 
 .. function:: fromstringlist(sequence, parser=None)