Skip to content

gh-108682: [Enum] raise TypeError if super().__new__ called in custom __new__ #108704

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 31, 2023
Merged

gh-108682: [Enum] raise TypeError if super().__new__ called in custom __new__ #108704

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
78b9f67
[Enum] raise TypeError if super().__new__ called in custom __new__
ethanfurman Aug 31, 2023
7693241
Update Doc/howto/enum.rst
ethanfurman Aug 31, 2023
561dac5
fix whitespace
ethanfurman Aug 31, 2023
0435142
proliferate __new__ warning
ethanfurman Aug 31, 2023
4e93222
remove warning, add link to detail section
ethanfurman Aug 31, 2023
1fbf624
fix reference
ethanfurman Aug 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion Doc/howto/enum.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -426,10 +426,17 @@ enumeration, with the exception of special methods (:meth:`__str__`,
:meth:`__add__`, etc.), descriptors (methods are also descriptors), and
variable names listed in :attr:`_ignore_`.

Note: if your enumeration defines :meth:`__new__` and/or :meth:`__init__` then
Note: if your enumeration defines :meth:`__new__` and/or :meth:`__init__`,
any value(s) given to the enum member will be passed into those methods.
See `Planet`_ for an example.

.. note::

The :meth:`__new__` method, if defined, is used during creation of the Enum
members; it is then replaced by Enum's :meth:`__new__` which is used after
class creation for lookup of existing members. See :ref:`new-vs-init` for
more details.


Restricted Enum subclassing
---------------------------
Expand Down Expand Up @@ -895,6 +902,8 @@ Some rules:
:meth:`__str__` method has been reset to their data types'
:meth:`__str__` method.

.. _new-vs-init:

When to use :meth:`__new__` vs. :meth:`__init__`
------------------------------------------------

Expand Down Expand Up @@ -927,6 +936,11 @@ want one of them to be the value::
>>> print(Coordinate(3))
Coordinate.VY

.. warning::

*Do not* call ``super().__new__()``, as the lookup-only ``__new__`` is the one
that is found; instead, use the data type directly.


Finer Points
^^^^^^^^^^^^
Expand Down Expand Up @@ -1353,6 +1367,13 @@ to handle any extra arguments::
members; it is then replaced by Enum's :meth:`__new__` which is used after
class creation for lookup of existing members.

.. warning::

*Do not* call ``super().__new__()``, as the lookup-only ``__new__`` is the one
that is found; instead, use the data type directly -- e.g.::

obj = int.__new__(cls, value)


OrderedEnum
^^^^^^^^^^^
Expand Down
7 changes: 7 additions & 0 deletions Lib/enum.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -856,6 +856,8 @@ def _create_(cls, class_name, names, *, module=None, qualname=None, type=None, s
value = first_enum._generate_next_value_(name, start, count, last_values[:])
last_values.append(value)
names.append((name, value))
if names is None:
names = ()

# Here, names is either an iterable of (name, value) or a mapping.
for item in names:
Expand Down Expand Up @@ -1112,6 +1114,11 @@ def __new__(cls, value):
for member in cls._member_map_.values():
if member._value_ == value:
return member
# still not found -- verify that members exist, in-case somebody got here mistakenly
# (such as via super when trying to override __new__)
if not cls._member_map_:
raise TypeError("%r has no members defined" % cls)
#
Comment on lines +1117 to +1121
Copy link

@tyronexj tyronexj Oct 18, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it ok to move _member_map_ check after _missing_ hook?
I have a derived class has no members and hook _missing_, it raise exception here.
@ethanfurman

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, no. I tried to do that, but then the bug wasn't fixed.

What does your missing() hook do?

If you have no other alternative, you'll need to subclass EnumType and override the __call__ method.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick reply. here is an example:

from enum import Enum

class DynamicEnum(Enum):
    @classmethod
    def _missing_(cls, value):
        temp = object.__new__(cls)
        temp._name_ = f"{cls.__name__}_{value}"
        temp._value_ = value
        return temp

dynamic_enum = DynamicEnum(3)
print(dynamic_enum)
print(dynamic_enum.value)

which prints

DynamicEnum.DynamicEnum_3
3

before this change.

As my understanding, hook _missing_ handle the missing input, a dynamic value in the above case.
so I raise the question: hook _missing_ before empty check.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll chime in here as well. I have the same question and got hit by this change as well. We have a setup where we do like:

from enum import Enum

class Result(Enum):
    @classmethod
    def _missing_(cls, value):
        v = None
        for subclass in cls.__subclasses__():
            try:
                v = subclass(value)
            except:
                pass

        return v


class CommonResult(Result):
    OK = 0
    ERR_FOO = -0x1
    ERR_BAR = -0x2

Previously, Result(0) returned CommonResult.OK. Now throws. It was a nifty pattern to use, and it sort of breaks the combination of using the _missing_ and subclasses of Enum.

# still not found -- try _missing_ hook
try:
exc = None
Expand Down
Loading