[typing: PEP 646]: *tuple[int, int] is improperly evaluated by get_type_hints

[hjalmarlucius]

Bug report

Unpack information of typing.GenericAlias is not transferred from string annotations to interpreted annotations. typing.Unpacked works as expected.

A fix could be to add if getattr(t, "__unpacked__", False): return next(iter(GenericAlias(t.__origin__, ev_args))) clause here:

cpython/Lib/typing.py

Lines 373 to 374 in 6492492

	if isinstance(t, GenericAlias):
	return GenericAlias(t.__origin__, ev_args)

NOTE: This bug is in typing's internal API and I don't think there's issues in the public API (but haven't looked hard). However, this issue pops up quickly if you try to do anything wrt PEP 646 typing at runtime.

Minimal example

from typing import ForwardRef
from typing import TypeVarTuple

Ts = TypeVarTuple("Ts")
typ = ForwardRef("*Ts")._evaluate({}, {"Ts": Ts}, frozenset())
assert typ == next(iter(Ts))  # <-- PASSES AS EXPECTED

typ = ForwardRef("*tuple[int]")._evaluate({}, {}, frozenset())
assert typ == tuple[int]  # <-- PASSES BUT SHOULDN'T
assert typ == next(iter(tuple[int]))  # <-- SHOULD PASS BUT DOESN'T

Your environment

Python 3.11

Linked PRs

• gh-101015: Fix typing.get_type_hints with unpacked *tuple (PEP 646) #101031
• [3.11] gh-101015: Fix typing.get_type_hints with unpacked *tuple (PEP 646) (GH-101031) #101254

[sobolevn]

It happens because of this line:

cpython/Lib/typing.py

Lines 827 to 831 in ef633e5

	# If we do `def f(*args: *Ts)`, then we'll have `arg = '*Ts'`.
	# Unfortunately, this isn't a valid expression on its own, so we
	# do the unpacking manually.
	if arg[0] == '*':
	arg_to_compile = f'({arg},)[0]' # E.g. (*Ts,)[0]

Moreover, it is not ever tested 😒
If I delete this line all tests pass.

Moreover, it has different semantic for *tuple[int, str] and *Ts, as you have shown.
This happens because TypeVarTuple automatically adds Unpack here:

cpython/Lib/typing.py

Lines 1056 to 1057 in ef633e5

	def __iter__(self):
	yield Unpack[self]

I've sent a proposal with the fix, but I am not 100% happy with it. Please, share your feedback!

[AlexWaygood]

Can somebody provide an example of the bug that doesn't use private APIs such as typing._eval_type or ForwardRef._evaluate?

[sobolevn]

See my PR:

def test_get_type_hints_on_unpack_args(self):
        Ts = TypeVarTuple('Ts')

        def func2(*args: '*tuple[int, str]'): pass
        self.assertEqual(gth(func2), {'args': Unpack[tuple[int, str]]})

It was {'args': tuple[int, str]} before, example from the current main:

(.venv) ~/Desktop/cpython  main ✔                                                    
» ./python.exe                                                             
Python 3.12.0a4+ (heads/issue-100942-dirty:a2babb4f58, Jan 12 2023, 01:08:31) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import *
>>> Ts = TypeVarTuple('Ts')
>>> def func2(*args: '*tuple[int, str]'): pass
... 
>>> get_type_hints(func2)
{'args': tuple[int, str]}

[AlexWaygood]

Thanks @sobolevn, that's very helpful! Cc. @mrahtz for interest.

[AlexWaygood]

The bug is not unique to forward references. Here, obtaining the annotations directly via .__annotations__ gives the correct result, but going via get_type_hints gives the incorrect result. No forward references here:

>>> import typing
>>> def foo(*args: *tuple[int, int]): ...
>>> foo.__annotations__
{'args': *tuple[int, int]}
>>> typing.get_type_hints(foo)
{'args': tuple[int, int]}

[sobolevn]

Yes, that's a good catch. My PR does not solve the second problem. Fixing it!

[sobolevn]

Now it does. With even more hacks 😒

[hjalmarlucius]

I don't have a setup to test changes so going by intuition to simplify the change: If you try only the fix I proposed in _eval_type using next(iter(...)), are there still errors? ForwardRef.__forward_value__ comes from _eval_type and if the latter returns a correctly unpacked type, it shouldn't be necessary to re-apply it or introducing is_unpack.

[sobolevn]

@hjalmarlucius I've managed to remove new ForwardRef flag. However, next(iter(...)) seems like a very unclear hack :)

I prefer to keep my code with __unpacked__.

[AlexWaygood]

Thanks @hjalmarlucius for the report, and @sobolevn for the fix!