3.14 regression: Union[int, str] is Union[int, str] does not hold

[hroncok]

Bug report

Bug description:

Python 3.13.2

>>> from typing import Union
>>> Union[int, str] is Union[int, str]
True

Python 3.14.0a6

>>> from typing import Union
>>> Union[int, str] is Union[int, str]
False

Is this change desired, is it documented? This breaks tests in https://github.com/python-attrs/cattrs/blob/d8f17b7e7e57006171027df4d647a6fe068b623e/tests/test_disambiguators.py#L211

assert fn({"op": 0, "t": "MESSAGE_CREATE"}) is Union[F, G]

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

• gh-131933: Document UnionType/Union merger in What's New #131941

[hugovk]

Bisects to dc6d66f, PR #105511, issue #105499: "Merge typing.Union and types.UnionType".

[picnixz]

cc @JelleZijlstra

[JelleZijlstra]

I think that test is incorrect and I posted python-attrs/cattrs#642 to fix it. We never guaranteed anything about the behavior of is with Union.

This does indicate a possible real-world issue that I hadn't considered, though. The Union[A, B] is Union[A, B] property held on earlier versions because we had a cache that deduplicated these objects, and that cache was added to avoid excessive memory usage if a program contains many instances of the same union. So it's possible that removing this cache will cause increased memory usage for some programs.

We could re-add a cache to the C implementation of typing.Union now. But here's a few reasons why I'm not sure that's necessary:

• Python 3.14 will also have deferred evaluation of annotations, meaning many programs will contain a lot fewer instances of Union objects.
• Adding the cache will have some cost too. It will obviously make union construction slower (we'd have to construct the cache key and check the cache dict). It will also increase memory usage because we have to put the object in a cache dictionary. A Union object is 72 bytes; to store it in the cache we'll have to make a weakref to it (80 bytes) and add an entry to the dictionary (which appears to add about 33 bytes per key on average). We also need to save the cache key, which likely means a new tuple because we have to use @lru_cache(typed=True) or a moral equivalent; that's likely another 72 bytes or so. So if a particular union only appears in a program three or fewer times, we likely lose memory by adding the cache.
• New-style unions (int | str) don't do the caching and never did (int | str is int | str was False even on 3.13). As far as I know nobody has complained about this, and several users running large Python codebases have already migrated to this style of union.

This makes me think that it's not worth re-adding the cache system that would make it so running Union[int, str] twice gives you the same object.

We can add a note to the What's New for 3.14 though pointing out this behavior change.

[JelleZijlstra]

One more point around memory usage: new-style union objects are themselves much smaller. Old typing.Union had a __dict__, which was at least 272 bytes on 3.13; now on 3.14 the whole object is 72 bytes. Though admittedly the size of int | str got bigger from 3.13 to 3.14 (48 to 72 bytes) because we had to add some new pointers.

[JelleZijlstra]

Oh and one more thing (in case someone else complains about this later): the Union[int, str] is Union[int, str] identity wasn't very reliable in 3.13 and earlier anyway, because it was based on an LRU cache. Therefore, if you create enough other unions in between, you'll get two different objects:

>>> from typing import Literal, Union
>>> lst = [Union[int, str], *[Union[str, Literal[i]] for i in range(1000)], Union[int, str]]
>>> lst[0]
typing.Union[int, str]
>>> lst[-1]
typing.Union[int, str]
>>> lst[0] is lst[-1]
False