Inheriting Overloaded Signatures

[adam-grant-hendry]

Preamble

I have learned from

• My own mistakes
• mypy: Issue #5146
• mypy: Issue #10699

that @overloads are not inherited from superclasses and must be redefined in subclasses.

I've also learned there has been considerable discussion (mostly indirectly; initially stemmed from callback protocols) on the topic of trying to develop a way for users to declare that subclasses inherit the methods signatures of their parents:

• python typing: Issue #769
• python typing: Issue #269
• python typing: Issue #264
• python typing: Issue #239
• mypy: Issue #3028

I am trying to absorb all the information here. Although I thought I knew type hinting well, I'm realizing there is still so much I that don't know.

I'd rather not open up another Issue unnecessarily and complicate matters further, but I am still confused and have some questions.

Questions

1. Does anyone know if inheriting overloads is possible yet? (i.e. has a robust technique/recipe been developed yet and/or has some new functionality been added to the typing ecosystem to enable this?)

2. If yes, can someone please share with me (simply please! (if possible)) how to do this?

Benefits

To reiterate what has already been stated, this would be nice to have for the following reasons:

1. It's DRY, so
   (a) less code is typed
   (b) the chances of making a mistake where code is repeated is reduced
   (c) less code must be updated when new overloads need to be added
   though understandably, repeating overloads is more explicit

Research

From what I have gathered, the following approaches have been suggested:

1. @copy_type decorator

Source

from typing import Any, Callable, TypeVar

Ts = TypeVar('Ts')

# Implementation
def copy_type(f: Ts) -> Callable[[Any], Ts]:
    return lambda x: x

# Example Usage
@copy_type(open)
def backup_open(file: Any, *args: Any, **kwargs: Any) -> Any: ...

Pitfalls:

• Only enforces types on the method when it is invoked, but not within the function body definition itself

Thoughts?

Perhaps a Protocol can be used?

The overloaded function implementation does not implement all possible arguments error and LSP feel much more nominal subtypish to me, but perhaps there is a structural subtyping way to achieve this?

[erictraut]

I consider overloads an advanced feature. Very few Python developers, even among those who regularly use type annotations in their code, use (or even know about) @overload. Overloads are most useful for annotating legacy libraries that have polymorphic functions and methods. Developers who are annotating their own code or developing new libraries with static typing in mind tend to use functional polymorphism less frequently because typing is much more top of mind when they are writing the code. So I don't know that this is a big issue.

Overriding an overloaded method is even more uncommon. I can remember needing to do this only twice in as many years. So I don't think that overridden overloaded functions is a common problem.

The decision of whether to inherit type declarations (of attributes or methods) from a base class is not currently standardized, and behaviors vary from one type checker to the next. Some of this is rooted in the somewhat different philosophies of the type checkers. For example, mypy doesn't type check any functions that lack type annotations. Pyright always type checks all functions based on whatever available type information is present. If pyright encounters a method whose input parameters are not annotated, it will use types from a base class method of the same name, but only if the number and names of the parameters matches. Pylance (the Python language based on pyright) also supports completions when defining a new method within a derived class. If you start typing a method definition, it will offer to fill in the remainder of the signature (including type annotations) based on the base class method definition. It does not offer to complete overloads, however. To my knowledge, no pylance user has ever asked for completion of overloads, which further underscores my assertion above that this is a rare occurrence.

[adam-grant-hendry]

Thank you @erictraut !

For overriding, I try to stick to the traditional idiom of writing a from_ method (or @classmethod if it's for an alternate constructor). This makes more sense to me if I'm intentionally breaking LSP.

I do hit challenges with C libraries ported to Python, like Qt and VTK. These use a lot of polymorphic functions and present a significant challenge. Often, the Qt library requires you to subclass a base class in order to implement it, and these have many polymorphic functions. I also have to be careful because keyword arguments aren't natively supported in C++, so I have to be sure to use /, *, and/or double-underscore prefixed variables appropriately.

[adam-grant-hendry]

@erictraut Do you have any thoughts on type hinting the python docs LRU cache example?