Generic ClassVar Type failing

[alanhdu]

With mypy 0.600 (and the latest master eb1bb064d707ce05735ff6795df82126e75ea6ea), running mypy on

from typing import ClassVar, Type, Generic, TypeVar

_T = TypeVar('_T')

class Array(Generic[_T]):
    _length_: ClassVar[int]
    _type_: ClassVar[Type[_T]]

def generate() -> Type[Array[float]]:
    pass

generate()._type_(3)

fails with

test.py:12: error: Too many arguments for "object"

instead of allowing this to construct a float.

The original use-case here is in the ctypes stubs, where mypy fails to derive the type of (ctypes.c_float * 5)._type_ -- see:

https://github.com/python/typeshed/blob/9ec6d476c44729ea84e9cafde9c5a77610451b0c/stdlib/2and3/ctypes/__init__.pyi#L248-L250

[sutaner]

Maybe it has something to do with:

Note that a ClassVar parameter cannot include any type variables, regardless of the level of nesting: ClassVar[T] and ClassVar[List[Set[T]]] are both invalid if T is a type variable.

See PEP 526 (https://www.python.org/dev/peps/pep-0526/#class-and-instance-variable-annotations)

[ilevkivskyi]

One comment is now have good stubs and a plugin for ctypes, another comment is maybe this restriction in the PEP is to limiting, accessing generic class variables when they are still generic should indeed be prohibited, but declaring it and accessing only on subclasses that has that variable bound should be OK (technically not 100% safe but but I think should be fine).

[ktbarrett]

I agree the PEP does seem unnecessarily limiting. If you want to use a type variable at runtime, the idiom might look like:

class ListWithElementType(List[T], Generic[T]):
    element_type: ClassVar[Type[T]]
    # methods that use the element_type at runtime via type(self).element_type

class IntList(ListWithElementType[int]):
    element_type = int

ListWithElementType has an abstract attribute and trying to instantiate it should fail. Subclasses bind a value to the abstract attribute and wouldn't fail. An exception could be made specifically for ClassVar of TypeVar if it is abstract.

The following case should fail. Maybe this is what they had in mind when writing that rule.

class Bad(Generic[T]):
    # This would bound T at declaration, so this makes no sense.
    thing: ClassVar[T] = 0  

But it's even more interesting than that, it's possible bounding T is okay iff the type of the initializing expression is the same type as the lower bound of the TypeVar.

NumT = TypeVar("NumT", bound=Number)

class PossiblyBadButMaybeNotOneDay(Generic[NumT]):
    # This would not be correct if NumT were inferred to be an `int` subtype,
    # but it would be fine if it were inferred to be an `int` supertype like `float`, `complex`, etc.
    # So in a future with TypeVar lower bounds,
    # it would be okay if the lower bound was the same type as the initializer.
    thing: ClassVar[NumT] = 0

[michaeloliverx]

I agree the PEP does seem unnecessarily limiting. If you want to use a type variable at runtime, the idiom might look like:

class ListWithElementType(List[T], Generic[T]):
    element_type: ClassVar[Type[T]]
    # methods that use the element_type at runtime via type(self).element_type

class IntList(ListWithElementType[int]):
    element_type = int

This is exactly what I was looking to do, does the PEP need to be revised first for this to happen?

[embray]

Ditto--I'm not sure why this limitation was made in the first place. Of course it could be violated by an assignment expression, but for an abstract base class it's not such an unusual thing to do. I don't see any obvious reason in this case why

element_type: ClassVar[Type[T]]

should be different from

element_type: Type[T]

It's OK if it doesn't have a value--we'll still be giving it one later.

[bedlamzd]

Here just to say that I've stumbled into this problem when defining abstract class, so I'd be happy if this restriction is lifted. Do you know if there is any issue/proposal to revise related pep?

[Tinche]

I changed the attrs plugin recently to correctly generate the __attrs_attrs__ property as a ClassVar without knowing Mypy doesn't support generic ClassVars. So now the natural protocol definition for an attrs class doesn't work, which means the signature of fields() ((type[AttrsClass[T]] -> T) cannot be correct and I'm a sad panda.

(The proposed protocol:

_T = TypeVar("_T")

class AttrsClass(Protocol[_T]):
    __attrs_attrs__: ClassVar[_T]

)

[jchalupka-pan]

I believe that the reasoning given by @ktbarrett above makes sense. Anybody knows if there is any movement towards this? Any PEPs, discussions or anything?

[erictraut]

There has been some discussion in the python/typing discussion forum, but there hasn't been any recent movement toward eliminating this limitation. If you're interested in championing such a change, I recommend participating in the python/typing forums.

[caniko]

There has been some discussion in the python/typing#1424, but there hasn't been any recent movement toward eliminating this limitation.

@levkivskyi did comment on the matter earlier, and it does make sense. We need a PR to move forward.