gh-90815: Add mimalloc memory allocator

.gitignore

@@ -142,6 +142,10 @@ Tools/ssl/amd64
 Tools/ssl/win32
 Tools/freeze/test/outdir
 
+# unused mimalloc files
+Include/internal/mimalloc/mimalloc-new-delete.h
+Include/internal/mimalloc/mimalloc-override.h
+
 # The frozen modules are always generated by the build so we don't
 # keep them in the repo.  Also see Tools/scripts/freeze_modules.py.
 Python/frozen_modules/*.h

Doc/c-api/init_config.rst

@@ -243,15 +243,28 @@ PyPreConfig
       * ``PYMEM_ALLOCATOR_PYMALLOC_DEBUG`` (``6``): :ref:`Python pymalloc
         memory allocator <pymalloc>` with :ref:`debug hooks
         <pymem-debug-hooks>`.
+      * ``PYMEM_ALLOCATOR_MIMALLOC`` (``7``): :ref:`mimalloc
+        memory allocator <mimalloc>`.
+      * ``PYMEM_ALLOCATOR_MIMALLOC_DEBUG`` (``8``): :ref:`mimalloc
+        memory allocator <mimalloc>` with :ref:`debug hooks
+        <pymem-debug-hooks>`.
 
       ``PYMEM_ALLOCATOR_PYMALLOC`` and ``PYMEM_ALLOCATOR_PYMALLOC_DEBUG`` are
       not supported if Python is :option:`configured using --without-pymalloc
       <--without-pymalloc>`.
 
+      ``PYMEM_ALLOCATOR_MIMALLOC`` and ``PYMEM_ALLOCATOR_MIMALLOC_DEBUG`` are
+      not supported unless Python is :option:`configured using --with-mimalloc
+      <--with-mimalloc>`.
+
       See :ref:`Memory Management <memory>`.
 
       Default: ``PYMEM_ALLOCATOR_NOT_SET``.
 
+      .. versionchanged:: 3.11
+         Added ``PYMEM_ALLOCATOR_MIMALLOC`` and
+         ``PYMEM_ALLOCATOR_MIMALLOC_DEBUG``.
+
    .. c:member:: int configure_locale
 
       Set the LC_CTYPE locale to the user preferred locale?

Doc/c-api/memory.rst

@@ -377,10 +377,12 @@ Default memory allocators:
 ===============================  ====================  ==================  =====================  ====================
 Configuration                    Name                  PyMem_RawMalloc     PyMem_Malloc           PyObject_Malloc
 ===============================  ====================  ==================  =====================  ====================
-Release build                    ``"pymalloc"``        ``malloc``          ``pymalloc``           ``pymalloc``
-Debug build                      ``"pymalloc_debug"``  ``malloc`` + debug  ``pymalloc`` + debug   ``pymalloc`` + debug
-Release build, without pymalloc  ``"malloc"``          ``malloc``          ``malloc``             ``malloc``
-Debug build, without pymalloc    ``"malloc_debug"``    ``malloc`` + debug  ``malloc`` + debug     ``malloc`` + debug
+Release build, with mimalloc     ``"mimalloc"``        ``mimalloc``        ``mimalloc``           ``mimalloc``
+Debug build, with mimalloc       ``"mimalloc_debug"``  ``malloc`` + debug  ``mimalloc`` + debug   ``mimalloc`` + debug
+Release build, without mimalloc  ``"pymalloc"``        ``malloc``          ``pymalloc``           ``pymalloc``
+Debug build, without mimalloc    ``"pymalloc_debug"``  ``malloc`` + debug  ``pymalloc`` + debug   ``pymalloc`` + debug
+Release build, without both      ``"malloc"``          ``malloc``          ``malloc``             ``malloc``
+Debug build, without both        ``"malloc_debug"``    ``malloc`` + debug  ``malloc`` + debug     ``malloc`` + debug
 ===============================  ====================  ==================  =====================  ====================
 
 Legend:
@@ -389,6 +391,7 @@ Legend:
 * ``malloc``: system allocators from the standard C library, C functions:
   :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` and :c:func:`free`.
 * ``pymalloc``: :ref:`pymalloc memory allocator <pymalloc>`.
+* ``mimalloc``: :ref:`mimalloc memory allocator <mimalloc>`.
 * "+ debug": with :ref:`debug hooks on the Python memory allocators
   <pymem-debug-hooks>`.
 * "Debug build": :ref:`Python build in debug mode <debug-build>`.
@@ -643,6 +646,21 @@ Customize pymalloc Arena Allocator
    Set the arena allocator.
 
 
+.. _mimalloc:
+
+The mimalloc allocator
+======================
+
+`mimalloc (pronounced "me-malloc") <https://github.com/microsoft/mimalloc>`_
+is a general purpose allocator with excellent performance characteristics.
+
+This allocator is disabled by default unless Python is configured with the
+:option:`--with-mimalloc` option. It can also be disabled at runtime using
+the :envvar:`PYTHONMALLOC` environment variable (ex: ``PYTHONMALLOC=malloc``).
+If Python is configured with both :ref:`pymalloc <pymalloc>` and mimalloc, then
+mimalloc is preferred.
+
+
 tracemalloc C API
 =================
 

Doc/using/cmdline.rst

@@ -885,13 +885,21 @@ conflict.
    * ``pymalloc``: use the :ref:`pymalloc allocator <pymalloc>` for
      :c:data:`PYMEM_DOMAIN_MEM` and :c:data:`PYMEM_DOMAIN_OBJ` domains and use
      the :c:func:`malloc` function for the :c:data:`PYMEM_DOMAIN_RAW` domain.
+   * ``mimalloc``: use the :ref:`mimalloc allocator <mimalloc>` for
+     all domains (in debug mode :c:data:`PYMEM_DOMAIN_RAW` uses :c:func:`malloc`
+     function).
+
 
    Install :ref:`debug hooks <pymem-debug-hooks>`:
 
    * ``debug``: install debug hooks on top of the :ref:`default memory
      allocators <default-memory-allocators>`.
    * ``malloc_debug``: same as ``malloc`` but also install debug hooks.
    * ``pymalloc_debug``: same as ``pymalloc`` but also install debug hooks.
+   * ``mimalloc_debug``: same as ``mimalloc`` but also install debug hooks.
+
+   .. versionchanged:: 3.11
+      Added the ``"mimalloc"`` and ``"mimalloc_debug"`` allocators
 
    .. versionchanged:: 3.7
       Added the ``"default"`` allocator.

Doc/using/configure.rst

@@ -262,6 +262,30 @@ also be used to improve performance.
 
    See also :envvar:`PYTHONMALLOC` environment variable.
 
+.. cmdoption:: --with-mimalloc
+
+   Enable :ref:`mimalloc <mimalloc>` memory allocator. mimalloc is enabled
+   by default when compiler and platform provide C11 ``stdatomic.h``.
+
+   See also :envvar:`PYTHONMALLOC` environment variable.
+
+   .. versionadded:: 3.11
+
+.. cmdoption:: --enable-mimalloc-secure[=yes|no|1|2|3|4]
+
+   Enable mimalloc's secure mode and various mitigations against exploits.
+   Secure mode comes with small performance penalty and uses additional
+   memory for guard pages. Each level includes the previous levels. *yes*
+   enables the highest security level.
+
+   * *1* enables guard pages around metadata
+   * *2* enables guard pages around mimalloc page
+   * *3* enables encoded free lists and detects corrupted free lists as
+     well as invalid pointer frees.
+   * *4* enables expensive checks for double free.
+
+   .. versionadded:: 3.11
+
 .. cmdoption:: --without-doc-strings
 
    Disable static documentation strings to reduce the memory footprint (enabled

Include/cpython/pymem.h

@@ -41,6 +41,10 @@ typedef enum {
     PYMEM_ALLOCATOR_PYMALLOC = 5,
     PYMEM_ALLOCATOR_PYMALLOC_DEBUG = 6,
 #endif
+#ifdef WITH_MIMALLOC
+    PYMEM_ALLOCATOR_MIMALLOC = 7,
+    PYMEM_ALLOCATOR_MIMALLOC_DEBUG = 8,
+#endif
 } PyMemAllocatorName;