From f5205c67e2ad239d77fe25514cf423084ec64ac0 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Wed, 2 Aug 2023 14:40:40 -0700 Subject: [PATCH 01/67] Update after Erik's feedback --- pep-0722.rst | 205 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 205 insertions(+) create mode 100644 pep-0722.rst diff --git a/pep-0722.rst b/pep-0722.rst new file mode 100644 index 00000000000..8f1cdb8192e --- /dev/null +++ b/pep-0722.rst @@ -0,0 +1,205 @@ +PEP: 722 +Title: Stricter Type Guards +Author: Rich Chiodo , Eric Traut , Erik De Bonte +Sponsor: +PEP-Delegate: +Discussions-To: https://github.com/python/typing/discussions/1013 +Status: Draft +Type: Standards Track +Topic: Typing +Content-Type: text/x-rst +Created: 28-Jul-2023 +Python-Version: 3.10 +Post-History: +Resolution: + + +Abstract +======== + +:pep:`647` created a special return type annotation ``TypeGuard`` that allowed +type checkers to narrow types. + +This PEP further refines :pep:`647` by allowing type checkers to narrow types +even further when a ``TypeGuard`` function returns false. + +Motivation +========== + +`TypeGuards `__ are used throughout Python libraries to allow a +type checker to narrow what the type of something is when the ``TypeGuard`` +returns true. + +.. code-block:: python + + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + # Type checkers can assume val is a 'str' in this branch + +However, in the ``else`` clause, :pep:`647` didn't prescribe what the type might +be: + +.. code-block:: python + + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + # Type checkers can assume val is a 'str' in this branch + else: + # Type here is not narrowed. It is still 'str | int' + + +This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype +of the type of the first input argument, then the false case can be further +narrowed. + +This changes the example above like so: + +.. code-block:: python + + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + # Type checkers can assume val is a 'str' in this branch + else: + # Type checkers can assume val is an 'int' in this branch + +Since the ``TypeGuard`` type (or output type) is a subtype of the input argument +type, a type checker can determine that the only possible type in the ``else`` +is the other type in the Union. In this example, it is safe to assume that if +``is_str`` returns false, then type of the ``val`` argument is an ``int``. + +Unsafe Narrowing +-------------------- + +There are cases where this further type narrowing is not possible. Here's an +example: + +.. code-block:: python + + def is_str_list(val: list[int | str]) -> TypeGuard[list[str]] + return all(isinstance(x, str) for x in val) + + def func(val: list[int | str]): + if is_str_list(val): + # Type checker assumes list[str] here + else: + # Type checker cannot assume list[int] here + +Since ``list`` is invariant, it doesn't have any subtypes. This means type +checkers cannot narrow the type to ``list[int]`` in the false case. +``list[str]`` is not a subtype of ``list[str | int]``. + +Type checkers should not assume any narrowing in the false case when the +``TypeGuard`` type is not a subtype of the input argument type. + +However, narrowing in the true case is still possible. In the example above, the +type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` +function returns true. + +Specification +============= + +This PEP requires no new changes to the language. It is merely modifying the +definition of ``TypeGuard`` for type checkers. Runtimes should already be +behaving in this way. + +Existing ``TypeGuard`` usage may change though, as described below. + + +Backwards Compatibility +======================= + +For preexisting code this should require no changes, but should simplify this +use case here: + +.. code-block:: python + + class A(): + pass + class B(): + pass + + def is_A(x: A | B) -> TypeGuard[A]: + return is_instance(x, A) + + + def is_B(x: A | B) -> TypeGuard[B]: + return is_instance(x, B) + + + def test(x: A | B): + if is_A(x): + # Do stuff assuming x is an 'A' + return + assert is_B(x) + + # Do stuff assuming x is a 'B' + return + + +This use case becomes this instead: + +.. code-block:: python + + class A(): + pass + class B(): + pass + + def is_A(x: A | B) -> TypeGuard[A]: + return is_instance(x, A) + + + def test(x: A | B): + if is_A(x): + # Do stuff assuming x is an 'A' + return + + # Do stuff assuming x is a 'B' + return + + +How to Teach This +================= + +The belief is that new users will assume this is how ``TypeGuard`` works in the +first place. Meaning this change should make ``TypeGuard`` easier to teach. + + +Reference Implementation +======================== + +A reference `implementation `__ of this idea exists in Pyright. + + +Rejected Ideas +============== + +Originally a new ``StrictTypeGuard`` construct was proposed. A +``StrictTypeGuard`` would be similar to to a ``TypeGuard`` except it would +explicitly state that output type was a subtype of the input type. Type checkers +would validate that the output type was a subtype of the input type. + +See this comment: `StrictTypeGuard proposal `__ + +This was rejected because for most cases it's not necessary. Most people assume +the negative case for ``TypeGuard`` anyway, so why not just change the +specification to match their assumptions? + +Footnotes +========= +.. _typeguards: https://peps.python.org/pep-0647/ + +Copyright +========= + +This document is placed in the public domain or under the CC0-1.0-Universal +license, whichever is more permissive. \ No newline at end of file From 9da9233ea40b70a61d862c9757e118cc95c21d6d Mon Sep 17 00:00:00 2001 From: rchiodo Date: Wed, 2 Aug 2023 14:59:11 -0700 Subject: [PATCH 02/67] Erik's initial comments --- pep-0722.rst | 193 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 193 insertions(+) create mode 100644 pep-0722.rst diff --git a/pep-0722.rst b/pep-0722.rst new file mode 100644 index 00000000000..98cfbf1d2dd --- /dev/null +++ b/pep-0722.rst @@ -0,0 +1,193 @@ +PEP: 722 +Title: Stricter Type Guards +Author: Rich Chiodo , Eric Traut +Sponsor: +PEP-Delegate: +Discussions-To: https://github.com/python/typing/discussions/1013 +Status: Draft +Type: Standards Track +Topic: Typing +Content-Type: text/x-rst +Created: 28-Jul-2023 +Python-Version: 3.10 +Post-History: +Resolution: + + +Abstract +======== + +This PEP further refines `TypeGuards `__ to +indicate when negative type narrowing is deemed safe. + +[I'd suggest mentioning PEP 647 explicitly here rather than having the opaque link. You can link to a PEP in RST using :pep:`647` ] +[I think more context is needed here for readers to understand what "negative" means. Maybe one sentence explaining what typeguards currently do and then another about the negative issue.] + + +Motivation +========== + +`TypeGuards `__ are used throughout python +libraries but cannot be used to determine the negative case: + +[Again, more context is needed for the user to understand what "the negative case" means.] +[Also what does "determine the negative case" mean? Maybe something like "narrow the type in the negative case" would be more clear? Also see the use of that phrase below the code block.] +[python should be capitalized] + +:: +[I'm wondering if `::` is equivalent to `.. code-block:: python` -- You may need the latter to get proper colorization. Check after you build your RST to HTML.] + + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + reveal_type(val) # str + else: + reveal_type(val) # str | int + +This inability to determine the negative case makes ``TypeGuard`` not as useful as +it could be. + +This PEP proposes that in cases where the output type is a *strict* subtype of +the input type, the negative case can be computed. This changes the example so +that the ``int`` case is possible: +["output type" -- might need to define this term or use something else. I don't see that term used in PEP 647.] +["This changes the example" -- maybe rephrase this to clarify that the code of the example is unchanged, but type checkers can interpret it differently?] +["is possible" seems pretty vague] +[What does strict subtype mean? And why is it italicized?] + +:: + + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + reveal_type(val) # str + else: + reveal_type(val) # int + +Since the output type is a *strict* subtype of the +input, a type checker can determine that the only possible type in the ``else`` is the +other input type(s). +["the other input type(s)" -- There's only one input type. It's a Union. Suggest rephrasing this. I'm not sure if talking about the types using set theory (input -- output) would make this more clear (or more generic) or worse.] + +If the output type is not a *strict* subtype of the input type, +the negative cannot be assumed to be the intuitive opposite: +["intuitive opposite" -- opposite is the incorrect term here and I think intuition doesn't belong in a PEP :)] + +:: + + def is_str_list(val: list[int | str]) -> TypeGuard[list[str]] + return all(isinstance(x, str) for x in val) + + def func(val: list[int | str]): + if is_str_list(val): + reveal_type(val) # list[str] + else: + reveal_type(val) # list[str | int] + +Since ``list`` is invariant, it doesn't have any subtypes, so type checkers +can't narrow the type in the negative case. + +Specification +============= + +This PEP requires no new changes to the language. It is merely modifying the +definition of ``TypeGuard`` for type checkers. The runtime should already be +behaving in this way. +["should" -- "The runtime" sounds singular, so if you mean CPython alone, I'd remove "should". If you mean that all Python runtimes should be behaving this way, I'd clarify that.] + +Existing ``TypeGuard`` usage may change though, as described below. + + +Backwards Compatibility +======================= + +For preexisting code this should require no changes, but should simplify this +use case here: + +:: + + A = TypeVar("A") + B = TypeVar("B") + + def is_A(x: A | B) -> TypeGuard[A]: + raise NotImplementedError + + + def after_is_A(x: A | B) -> TypeGuard[B]: + return True + + + def test(x: A | B): + if is_A(x): + reveal_type(x) + return + assert after_is_A(x) + + reveal_type(x) + return + +["after_is_A" is confusing me -- is there a better name? "is_not_A"?] +[Can/should you use PEP 695 syntax for the TypeVars?] + +becomes this instead +["becomes this instead" is not a grammatically correct continuation of the sentence before the first code block. Maybe rephrase the sentence to "Preexisting code should require no changes, but code like this...can be simplified to this:"] +[Add comments in these code blocks showing the expected inferred type as you did above? I think then you won't need the reveal_type calls?] + +:: + + A = TypeVar("A") + B = TypeVar("B") + + def is_A(x: A | B) -> TypeGuard[A]: + return isinstance(x, A) + + + def test(x: A | B): + if is_A(x): + reveal_type(x) + return + reveal_type(x) + return + + +How to Teach This +================= + +The belief is that new users will assume this is how ``TypeGuard`` works in the +first place. Meaning this change should make ``TypeGuard`` easier to teach. + + +Reference Implementation +======================== + +A reference implementation of this idea exists in Pyright. +[Would there be value in pointing the reader to the implementation?] + + +Rejected Ideas +============== + +Originally a new ``StrictTypeGuard`` construct was proposed. A +``StrictTypeGuard`` would be similar to to a ``TypeGuard`` except it would +explicitly state that output type was a subtype of the input type. Type checkers +would validate that the output type was a subtype of the input type. + +See this comment: `StrictTypeGuard proposal `__ + +This was rejected because for most cases it's not necessary. Most people assume +the negative case for ``TypeGuard`` anyway, so why not just change the specification +to match their assumptions? + +Footnotes +========= +.. _typeguards: https://peps.python.org/pep-0647/ + +Copyright +========= + +This document is placed in the public domain or under the CC0-1.0-Universal +license, whichever is more permissive. \ No newline at end of file From 00d524744efa49df31574593d4b686caa721582b Mon Sep 17 00:00:00 2001 From: rchiodo Date: Wed, 2 Aug 2023 14:40:40 -0700 Subject: [PATCH 03/67] Update after Erik's feedback --- pep-0722.rst | 142 ++++++++++++++++++++++++++++----------------------- 1 file changed, 77 insertions(+), 65 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 98cfbf1d2dd..8f1cdb8192e 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -1,6 +1,6 @@ PEP: 722 Title: Stricter Type Guards -Author: Rich Chiodo , Eric Traut +Author: Rich Chiodo , Eric Traut , Erik De Bonte Sponsor: PEP-Delegate: Discussions-To: https://github.com/python/typing/discussions/1013 @@ -17,87 +17,99 @@ Resolution: Abstract ======== -This PEP further refines `TypeGuards `__ to -indicate when negative type narrowing is deemed safe. - -[I'd suggest mentioning PEP 647 explicitly here rather than having the opaque link. You can link to a PEP in RST using :pep:`647` ] -[I think more context is needed here for readers to understand what "negative" means. Maybe one sentence explaining what typeguards currently do and then another about the negative issue.] +:pep:`647` created a special return type annotation ``TypeGuard`` that allowed +type checkers to narrow types. +This PEP further refines :pep:`647` by allowing type checkers to narrow types +even further when a ``TypeGuard`` function returns false. Motivation ========== -`TypeGuards `__ are used throughout python -libraries but cannot be used to determine the negative case: +`TypeGuards `__ are used throughout Python libraries to allow a +type checker to narrow what the type of something is when the ``TypeGuard`` +returns true. + +.. code-block:: python -[Again, more context is needed for the user to understand what "the negative case" means.] -[Also what does "determine the negative case" mean? Maybe something like "narrow the type in the negative case" would be more clear? Also see the use of that phrase below the code block.] -[python should be capitalized] + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) + + def func(val: str | int): + if is_str(val): + # Type checkers can assume val is a 'str' in this branch -:: -[I'm wondering if `::` is equivalent to `.. code-block:: python` -- You may need the latter to get proper colorization. Check after you build your RST to HTML.] +However, in the ``else`` clause, :pep:`647` didn't prescribe what the type might +be: + +.. code-block:: python def is_str(val: str | int) -> TypeGuard[str]: return isinstance(val, str) def func(val: str | int): if is_str(val): - reveal_type(val) # str + # Type checkers can assume val is a 'str' in this branch else: - reveal_type(val) # str | int + # Type here is not narrowed. It is still 'str | int' + -This inability to determine the negative case makes ``TypeGuard`` not as useful as -it could be. +This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype +of the type of the first input argument, then the false case can be further +narrowed. -This PEP proposes that in cases where the output type is a *strict* subtype of -the input type, the negative case can be computed. This changes the example so -that the ``int`` case is possible: -["output type" -- might need to define this term or use something else. I don't see that term used in PEP 647.] -["This changes the example" -- maybe rephrase this to clarify that the code of the example is unchanged, but type checkers can interpret it differently?] -["is possible" seems pretty vague] -[What does strict subtype mean? And why is it italicized?] +This changes the example above like so: -:: +.. code-block:: python def is_str(val: str | int) -> TypeGuard[str]: return isinstance(val, str) def func(val: str | int): if is_str(val): - reveal_type(val) # str + # Type checkers can assume val is a 'str' in this branch else: - reveal_type(val) # int + # Type checkers can assume val is an 'int' in this branch + +Since the ``TypeGuard`` type (or output type) is a subtype of the input argument +type, a type checker can determine that the only possible type in the ``else`` +is the other type in the Union. In this example, it is safe to assume that if +``is_str`` returns false, then type of the ``val`` argument is an ``int``. -Since the output type is a *strict* subtype of the -input, a type checker can determine that the only possible type in the ``else`` is the -other input type(s). -["the other input type(s)" -- There's only one input type. It's a Union. Suggest rephrasing this. I'm not sure if talking about the types using set theory (input -- output) would make this more clear (or more generic) or worse.] +Unsafe Narrowing +-------------------- -If the output type is not a *strict* subtype of the input type, -the negative cannot be assumed to be the intuitive opposite: -["intuitive opposite" -- opposite is the incorrect term here and I think intuition doesn't belong in a PEP :)] +There are cases where this further type narrowing is not possible. Here's an +example: -:: +.. code-block:: python def is_str_list(val: list[int | str]) -> TypeGuard[list[str]] return all(isinstance(x, str) for x in val) def func(val: list[int | str]): if is_str_list(val): - reveal_type(val) # list[str] + # Type checker assumes list[str] here else: - reveal_type(val) # list[str | int] + # Type checker cannot assume list[int] here -Since ``list`` is invariant, it doesn't have any subtypes, so type checkers -can't narrow the type in the negative case. +Since ``list`` is invariant, it doesn't have any subtypes. This means type +checkers cannot narrow the type to ``list[int]`` in the false case. +``list[str]`` is not a subtype of ``list[str | int]``. + +Type checkers should not assume any narrowing in the false case when the +``TypeGuard`` type is not a subtype of the input argument type. + +However, narrowing in the true case is still possible. In the example above, the +type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` +function returns true. Specification ============= This PEP requires no new changes to the language. It is merely modifying the -definition of ``TypeGuard`` for type checkers. The runtime should already be +definition of ``TypeGuard`` for type checkers. Runtimes should already be behaving in this way. -["should" -- "The runtime" sounds singular, so if you mean CPython alone, I'd remove "should". If you mean that all Python runtimes should be behaving this way, I'd clarify that.] Existing ``TypeGuard`` usage may change though, as described below. @@ -108,49 +120,50 @@ Backwards Compatibility For preexisting code this should require no changes, but should simplify this use case here: -:: +.. code-block:: python - A = TypeVar("A") - B = TypeVar("B") + class A(): + pass + class B(): + pass def is_A(x: A | B) -> TypeGuard[A]: - raise NotImplementedError + return is_instance(x, A) - def after_is_A(x: A | B) -> TypeGuard[B]: - return True + def is_B(x: A | B) -> TypeGuard[B]: + return is_instance(x, B) def test(x: A | B): if is_A(x): - reveal_type(x) + # Do stuff assuming x is an 'A' return - assert after_is_A(x) + assert is_B(x) - reveal_type(x) + # Do stuff assuming x is a 'B' return -["after_is_A" is confusing me -- is there a better name? "is_not_A"?] -[Can/should you use PEP 695 syntax for the TypeVars?] -becomes this instead -["becomes this instead" is not a grammatically correct continuation of the sentence before the first code block. Maybe rephrase the sentence to "Preexisting code should require no changes, but code like this...can be simplified to this:"] -[Add comments in these code blocks showing the expected inferred type as you did above? I think then you won't need the reveal_type calls?] +This use case becomes this instead: -:: +.. code-block:: python - A = TypeVar("A") - B = TypeVar("B") + class A(): + pass + class B(): + pass def is_A(x: A | B) -> TypeGuard[A]: - return isinstance(x, A) + return is_instance(x, A) def test(x: A | B): if is_A(x): - reveal_type(x) + # Do stuff assuming x is an 'A' return - reveal_type(x) + + # Do stuff assuming x is a 'B' return @@ -164,8 +177,7 @@ first place. Meaning this change should make ``TypeGuard`` easier to teach. Reference Implementation ======================== -A reference implementation of this idea exists in Pyright. -[Would there be value in pointing the reader to the implementation?] +A reference `implementation `__ of this idea exists in Pyright. Rejected Ideas @@ -179,8 +191,8 @@ would validate that the output type was a subtype of the input type. See this comment: `StrictTypeGuard proposal `__ This was rejected because for most cases it's not necessary. Most people assume -the negative case for ``TypeGuard`` anyway, so why not just change the specification -to match their assumptions? +the negative case for ``TypeGuard`` anyway, so why not just change the +specification to match their assumptions? Footnotes ========= From 4c4a0e916579eb118617c6b58a3923873c137cf7 Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 12:37:30 -0700 Subject: [PATCH 04/67] Update pep-0722.rst --- pep-0722.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 8f1cdb8192e..da475f8ae5d 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -108,8 +108,8 @@ Specification ============= This PEP requires no new changes to the language. It is merely modifying the -definition of ``TypeGuard`` for type checkers. Runtimes should already be -behaving in this way. +definition of ``TypeGuard`` for type checkers. Runtimes are already behaving +in this way. Existing ``TypeGuard`` usage may change though, as described below. @@ -202,4 +202,4 @@ Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal -license, whichever is more permissive. \ No newline at end of file +license, whichever is more permissive. From 4f90e9cb3bae1b2c3d14aa5845489a76ab3a1f2e Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 13:17:28 -0700 Subject: [PATCH 05/67] Add the deliberate mistake case --- pep-0722.rst | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/pep-0722.rst b/pep-0722.rst index 8f1cdb8192e..d89dd9b2724 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -104,6 +104,43 @@ However, narrowing in the true case is still possible. In the example above, the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` function returns true. +Creating invalid narrowing +-------------------------- + +The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an example: + +.. code-block:: python + + def is_positive_int(val: int | str) -> TypeGuard[int]: + return isinstance(val, int) and val > 0 + + def func(val: int | str): + if is_positive_int(val): + # Type checker assumes int here + else: + # Type checker assumes str here + +A type checker will assume for the else case that the value is ``str``. This +is a change in behavior from :pep:`647` but as that pep stated `here `__ +there are many ways a determined or uninformed developer can subvert type safety. + +A better way to handle this example would be something like so: + +.. code-block:: python + + PosInt = NewType('PosInt', int) + + def is_positive_int(val: PosInt | int | str) -> TypeGuard[PosInt]: + return isinstance(val, int) and val > 0 + + def func(val: int | str): + if is_positive_int(val): + # Type checker assumes PosInt here + else: + # Type checker assumes str | int here + + + Specification ============= From f00e7a203542e9609811d690612163a5da8f0ef6 Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 13:57:35 -0700 Subject: [PATCH 06/67] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index f4095ece657..d5482f316e0 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -17,8 +17,7 @@ Resolution: Abstract ======== -:pep:`647` created a special return type annotation ``TypeGuard`` that allowed -type checkers to narrow types. +:pep:`647` introduced the concept of ``TypeGuard`` functions which return true if their input parameter matches their target type. For example, a function that returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's input parameter is a ``str``. This allows type checkers to narrow types in this positive case. This PEP further refines :pep:`647` by allowing type checkers to narrow types even further when a ``TypeGuard`` function returns false. From c895a0adb447f56dcb6a65f232caad43a0337b3d Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 13:57:52 -0700 Subject: [PATCH 07/67] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index d5482f316e0..e0c64cf6929 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -26,7 +26,7 @@ Motivation ========== `TypeGuards `__ are used throughout Python libraries to allow a -type checker to narrow what the type of something is when the ``TypeGuard`` +type checker to narrow the type of something when the ``TypeGuard`` returns true. .. code-block:: python From 5305af45f8aeadc759ee017a8acff535d680007d Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 13:58:52 -0700 Subject: [PATCH 08/67] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index e0c64cf6929..ff83d5690c5 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -92,9 +92,8 @@ example: else: # Type checker cannot assume list[int] here -Since ``list`` is invariant, it doesn't have any subtypes. This means type +Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not a subtype of ``list[str | int]``. This means type checkers cannot narrow the type to ``list[int]`` in the false case. -``list[str]`` is not a subtype of ``list[str | int]``. Type checkers should not assume any narrowing in the false case when the ``TypeGuard`` type is not a subtype of the input argument type. From b6335f07e73acd362b2f009825f49892f42eabad Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 13:59:01 -0700 Subject: [PATCH 09/67] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index ff83d5690c5..5b716a8c2d2 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -152,8 +152,8 @@ Existing ``TypeGuard`` usage may change though, as described below. Backwards Compatibility ======================= -For preexisting code this should require no changes, but should simplify this -use case here: +For preexisting code this should require no changes, but will allow +use cases such as the one below to be simplified: .. code-block:: python From f24ca7cfbb9e8f54a0ea760621227a18db58c55f Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 14:02:17 -0700 Subject: [PATCH 10/67] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index 5b716a8c2d2..6024b56c68a 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -96,7 +96,7 @@ Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not checkers cannot narrow the type to ``list[int]`` in the false case. Type checkers should not assume any narrowing in the false case when the -``TypeGuard`` type is not a subtype of the input argument type. +``TypeGuard`` type argument is not a subtype of the input argument type. However, narrowing in the true case is still possible. In the example above, the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` From cbf907a560825bae3d7a2973404256d958edaeb9 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 14:03:14 -0700 Subject: [PATCH 11/67] More review feedback --- pep-0722.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index 5b716a8c2d2..205d770bca8 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -180,7 +180,8 @@ use cases such as the one below to be simplified: return -This use case becomes this instead: +With this proposed change, the code above continues to work but could be +simplified by removing the assertion that x is of type B in the negative case: .. code-block:: python From e5a71ea171e7a06bbe6067e6311f8889ed97c4b7 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 14:03:56 -0700 Subject: [PATCH 12/67] More review feedback --- pep-0722.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index 8a607e7a820..254dab19c3e 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -72,7 +72,7 @@ This changes the example above like so: Since the ``TypeGuard`` type (or output type) is a subtype of the input argument type, a type checker can determine that the only possible type in the ``else`` -is the other type in the Union. In this example, it is safe to assume that if +is the other type in the ``Union``. In this example, it is safe to assume that if ``is_str`` returns false, then type of the ``val`` argument is an ``int``. Unsafe Narrowing From a32efa40d3acd3692736d5b0c0e4ad30c88d6bfd Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 14:10:58 -0700 Subject: [PATCH 13/67] Fix 80 column limit --- pep-0722.rst | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 254dab19c3e..e39a03d3905 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -17,7 +17,11 @@ Resolution: Abstract ======== -:pep:`647` introduced the concept of ``TypeGuard`` functions which return true if their input parameter matches their target type. For example, a function that returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's input parameter is a ``str``. This allows type checkers to narrow types in this positive case. +:pep:`647` introduced the concept of ``TypeGuard`` functions which return true +if their input parameter matches their target type. For example, a function that +returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's +input parameter is a ``str``. This allows type checkers to narrow types in this +positive case. This PEP further refines :pep:`647` by allowing type checkers to narrow types even further when a ``TypeGuard`` function returns false. @@ -72,8 +76,8 @@ This changes the example above like so: Since the ``TypeGuard`` type (or output type) is a subtype of the input argument type, a type checker can determine that the only possible type in the ``else`` -is the other type in the ``Union``. In this example, it is safe to assume that if -``is_str`` returns false, then type of the ``val`` argument is an ``int``. +is the other type in the ``Union``. In this example, it is safe to assume that +if ``is_str`` returns false, then type of the ``val`` argument is an ``int``. Unsafe Narrowing -------------------- @@ -92,8 +96,9 @@ example: else: # Type checker cannot assume list[int] here -Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not a subtype of ``list[str | int]``. This means type -checkers cannot narrow the type to ``list[int]`` in the false case. +Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not +a subtype of ``list[str | int]``. This means type checkers cannot narrow the +type to ``list[int]`` in the false case. Type checkers should not assume any narrowing in the false case when the ``TypeGuard`` type argument is not a subtype of the input argument type. @@ -105,7 +110,8 @@ function returns true. Creating invalid narrowing -------------------------- -The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an example: +The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an +example: .. code-block:: python @@ -120,7 +126,8 @@ The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an ex A type checker will assume for the else case that the value is ``str``. This is a change in behavior from :pep:`647` but as that pep stated `here `__ -there are many ways a determined or uninformed developer can subvert type safety. +there are many ways a determined or uninformed developer can subvert +type safety. A better way to handle this example would be something like so: From 708f60c9cb21bb9a65d2c8887da9574f9c7d1fdc Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 14:55:48 -0700 Subject: [PATCH 14/67] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index e39a03d3905..2cf59fae259 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -23,8 +23,8 @@ returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's input parameter is a ``str``. This allows type checkers to narrow types in this positive case. -This PEP further refines :pep:`647` by allowing type checkers to narrow types -even further when a ``TypeGuard`` function returns false. +This PEP further refines :pep:`647` by allowing type checkers to also narrow types +when a ``TypeGuard`` function returns false. Motivation ========== From 662341faf516a111c96fc2e6a2a4cc658f914dc5 Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Thu, 3 Aug 2023 14:57:04 -0700 Subject: [PATCH 15/67] Update pep-0722.rst Co-authored-by: Erik De Bonte --- pep-0722.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0722.rst b/pep-0722.rst index 2cf59fae259..242d394924a 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -58,7 +58,7 @@ be: This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype -of the type of the first input argument, then the false case can be further +of the type of the first input parameter, then the false case can be further narrowed. This changes the example above like so: From 24b88bab29fc5e181513d42d56461a858677f28b Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 15:07:36 -0700 Subject: [PATCH 16/67] More feedback --- pep-0722.rst | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 242d394924a..1baa7302015 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -79,6 +79,18 @@ type, a type checker can determine that the only possible type in the ``else`` is the other type in the ``Union``. In this example, it is safe to assume that if ``is_str`` returns false, then type of the ``val`` argument is an ``int``. + +Specification +============= + +This PEP requires no new changes to the language. It is merely modifying the +definition of ``TypeGuard`` for type checkers. Runtimes are already behaving +in this way. + + +Existing ``TypeGuard`` usage may change though, as described in +`Backwards Compatibility`_. + Unsafe Narrowing -------------------- @@ -145,17 +157,6 @@ A better way to handle this example would be something like so: # Type checker assumes str | int here - -Specification -============= - -This PEP requires no new changes to the language. It is merely modifying the -definition of ``TypeGuard`` for type checkers. Runtimes are already behaving -in this way. - -Existing ``TypeGuard`` usage may change though, as described below. - - Backwards Compatibility ======================= From a6822195fe5ca01f895c05eaa778b14749914121 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 15:10:58 -0700 Subject: [PATCH 17/67] Review feedback --- pep-0722.rst | 9 --------- 1 file changed, 9 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 1baa7302015..0ee3c2d74bb 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -33,15 +33,6 @@ Motivation type checker to narrow the type of something when the ``TypeGuard`` returns true. -.. code-block:: python - - def is_str(val: str | int) -> TypeGuard[str]: - return isinstance(val, str) - - def func(val: str | int): - if is_str(val): - # Type checkers can assume val is a 'str' in this branch - However, in the ``else`` clause, :pep:`647` didn't prescribe what the type might be: From 6287be5c7dac1aeb9b7d9c326ed141e29270382b Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 15:36:46 -0700 Subject: [PATCH 18/67] Some more subtle word changes --- pep-0722.rst | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 0ee3c2d74bb..9a9a90d3176 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -49,8 +49,8 @@ be: This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype -of the type of the first input parameter, then the false case can be further -narrowed. +of the type of the first input parameter, then the false return case can be +further narrowed. This changes the example above like so: @@ -65,10 +65,11 @@ This changes the example above like so: else: # Type checkers can assume val is an 'int' in this branch -Since the ``TypeGuard`` type (or output type) is a subtype of the input argument -type, a type checker can determine that the only possible type in the ``else`` -is the other type in the ``Union``. In this example, it is safe to assume that -if ``is_str`` returns false, then type of the ``val`` argument is an ``int``. +Since the ``TypeGuard`` type (or output type) is a subtype of the first input +parameter type, a type checker can determine that the only possible type in the +``else`` is the other type in the ``Union``. In this example, it is safe to +assume that if ``is_str`` returns false, then type of the ``val`` argument is an +``int``. Specification @@ -104,7 +105,7 @@ a subtype of ``list[str | int]``. This means type checkers cannot narrow the type to ``list[int]`` in the false case. Type checkers should not assume any narrowing in the false case when the -``TypeGuard`` type argument is not a subtype of the input argument type. +``TypeGuard`` type argument is not a subtype of the first input parameter type. However, narrowing in the true case is still possible. In the example above, the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` From a2e0b22bce007ac1437ef3a1217244269cab4d50 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 3 Aug 2023 15:48:04 -0700 Subject: [PATCH 19/67] Change verbiage a little more --- pep-0722.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/pep-0722.rst b/pep-0722.rst index 9a9a90d3176..1d7502d22a6 100644 --- a/pep-0722.rst +++ b/pep-0722.rst @@ -86,8 +86,8 @@ Existing ``TypeGuard`` usage may change though, as described in Unsafe Narrowing -------------------- -There are cases where this further type narrowing is not possible. Here's an -example: +There are cases where this further type narrowing is not possible though. Here's +an example: .. code-block:: python @@ -111,7 +111,7 @@ However, narrowing in the true case is still possible. In the example above, the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` function returns true. -Creating invalid narrowing +User error -------------------------- The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an @@ -152,8 +152,9 @@ A better way to handle this example would be something like so: Backwards Compatibility ======================= -For preexisting code this should require no changes, but will allow -use cases such as the one below to be simplified: +For preexisting code this PEP should require no changes. + +However, some use cases such as the one below can be simplified: .. code-block:: python From 5850d66b96c153f5567746f4f6d4bcf80d706141 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 13:32:29 -0700 Subject: [PATCH 20/67] Change pep number --- pep-0722.rst => pep-0725.rst | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename pep-0722.rst => pep-0725.rst (100%) diff --git a/pep-0722.rst b/pep-0725.rst similarity index 100% rename from pep-0722.rst rename to pep-0725.rst From 641b761c76ff23bcc89b0b38fcf96da3de979d43 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 13:35:56 -0700 Subject: [PATCH 21/67] Update PEP number --- pep-0725.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0725.rst b/pep-0725.rst index 1d7502d22a6..e6f09189d79 100644 --- a/pep-0725.rst +++ b/pep-0725.rst @@ -1,4 +1,4 @@ -PEP: 722 +PEP: 725 Title: Stricter Type Guards Author: Rich Chiodo , Eric Traut , Erik De Bonte Sponsor: From 1ec22076c7722c87766851e41fdca6835b3d6986 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 13:51:38 -0700 Subject: [PATCH 22/67] Fix some errors in the pre-commit --- .github/CODEOWNERS | 1 + pep-0725.rst | 18 ++++++------------ 2 files changed, 7 insertions(+), 12 deletions(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index a41ddc148ce..7ac225c9839 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -600,6 +600,7 @@ pep-0718.rst @gvanrossum pep-0719.rst @Yhg1s pep-0720.rst @FFY00 pep-0721.rst @encukou +pep-0725.rst TBD # ... # pep-0754.txt # ... diff --git a/pep-0725.rst b/pep-0725.rst index e6f09189d79..0d8e4e47db7 100644 --- a/pep-0725.rst +++ b/pep-0725.rst @@ -3,7 +3,7 @@ Title: Stricter Type Guards Author: Rich Chiodo , Eric Traut , Erik De Bonte Sponsor: PEP-Delegate: -Discussions-To: https://github.com/python/typing/discussions/1013 +Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ Status: Draft Type: Standards Track Topic: Typing @@ -29,9 +29,8 @@ when a ``TypeGuard`` function returns false. Motivation ========== -`TypeGuards `__ are used throughout Python libraries to allow a -type checker to narrow the type of something when the ``TypeGuard`` -returns true. +``TypeGuards`` are used throughout Python libraries to allow a type checker to +narrow the type of something when the ``TypeGuard`` returns true. However, in the ``else`` clause, :pep:`647` didn't prescribe what the type might be: @@ -128,10 +127,9 @@ example: else: # Type checker assumes str here -A type checker will assume for the else case that the value is ``str``. This -is a change in behavior from :pep:`647` but as that pep stated `here `__ -there are many ways a determined or uninformed developer can subvert -type safety. +A type checker will assume for the else case that the value is ``str``. This is +a change in behavior from :pep:`647` but as that pep stated, there are many ways +a determined or uninformed developer can subvert type safety. A better way to handle this example would be something like so: @@ -231,10 +229,6 @@ This was rejected because for most cases it's not necessary. Most people assume the negative case for ``TypeGuard`` anyway, so why not just change the specification to match their assumptions? -Footnotes -========= -.. _typeguards: https://peps.python.org/pep-0647/ - Copyright ========= From 6184c243e62c10000c4e89f8bb97f7f07e693900 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 14:22:09 -0700 Subject: [PATCH 23/67] Review feedback --- pep-0725.rst => pep-0724.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) rename pep-0725.rst => pep-0724.rst (98%) diff --git a/pep-0725.rst b/pep-0724.rst similarity index 98% rename from pep-0725.rst rename to pep-0724.rst index 0d8e4e47db7..03149607c8f 100644 --- a/pep-0725.rst +++ b/pep-0724.rst @@ -1,15 +1,14 @@ -PEP: 725 +PEP: 724 Title: Stricter Type Guards Author: Rich Chiodo , Eric Traut , Erik De Bonte -Sponsor: -PEP-Delegate: +Sponsor: Jelle Zijlstra Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ Status: Draft Type: Standards Track Topic: Typing Content-Type: text/x-rst Created: 28-Jul-2023 -Python-Version: 3.10 +Python-Version: 3.13 Post-History: Resolution: From 7ab28d92c459b157c18aec244864e648a7bb81d0 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 14:23:36 -0700 Subject: [PATCH 24/67] Update codeowners --- .github/CODEOWNERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index ecd91864d09..c0db2c98c18 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -601,7 +601,7 @@ pep-0719.rst @Yhg1s pep-0720.rst @FFY00 pep-0721.rst @encukou pep-0722.rst @pfmoore -pep-0725.rst TBD +pep-0724.rst @jellezijlstra # ... # pep-0754.txt # ... From dddc7cb1ad3706abeaf2f7f1d860c36e86c2ce98 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Date: Fri, 4 Aug 2023 23:08:26 +0100 Subject: [PATCH 25/67] Remove ``Resolution`` --- pep-0724.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index 03149607c8f..233be11d8e0 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -10,7 +10,6 @@ Content-Type: text/x-rst Created: 28-Jul-2023 Python-Version: 3.13 Post-History: -Resolution: Abstract From 7e511590f91f73b466b262f816c8dea2d5cc50ca Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 15:08:36 -0700 Subject: [PATCH 26/67] Move to specification --- pep-0724.rst | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 03149607c8f..d6bfdc31bc2 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -46,6 +46,9 @@ be: # Type here is not narrowed. It is still 'str | int' +Specification +============= + This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype of the type of the first input parameter, then the false return case can be further narrowed. @@ -70,17 +73,6 @@ assume that if ``is_str`` returns false, then type of the ``val`` argument is an ``int``. -Specification -============= - -This PEP requires no new changes to the language. It is merely modifying the -definition of ``TypeGuard`` for type checkers. Runtimes are already behaving -in this way. - - -Existing ``TypeGuard`` usage may change though, as described in -`Backwards Compatibility`_. - Unsafe Narrowing -------------------- From a9b0407e0ad29d74dfc4ac125cd0398c8a22e51a Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Fri, 4 Aug 2023 15:24:20 -0700 Subject: [PATCH 27/67] Update pep-0724.rst Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- pep-0724.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index 3cdf616b35e..f6e212d6711 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -202,7 +202,9 @@ first place. Meaning this change should make ``TypeGuard`` easier to teach. Reference Implementation ======================== -A reference `implementation `__ of this idea exists in Pyright. +A reference `implementation`__ of this idea exists in Pyright. + +__ https://github.com/microsoft/pyright/commit/9a5af798d726bd0612cebee7223676c39cf0b9b0 Rejected Ideas From 106f10acfdd4041318ca7d855cb7e74d9a1d42a3 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 15:24:51 -0700 Subject: [PATCH 28/67] Formatting and grammar --- pep-0724.rst | 69 +++++++++++++++++++++++++++------------------------- 1 file changed, 36 insertions(+), 33 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 3cdf616b35e..7afb4de1f41 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -1,6 +1,8 @@ PEP: 724 Title: Stricter Type Guards -Author: Rich Chiodo , Eric Traut , Erik De Bonte +Author: Rich Chiodo , + Eric Traut , + Erik De Bonte Sponsor: Jelle Zijlstra Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ Status: Draft @@ -15,23 +17,23 @@ Post-History: Abstract ======== -:pep:`647` introduced the concept of ``TypeGuard`` functions which return true -if their input parameter matches their target type. For example, a function that -returns ``TypeGuard[str]`` is assumed to return ``true`` if and only if it's -input parameter is a ``str``. This allows type checkers to narrow types in this -positive case. +:pep:`647` introduced the concept of ``TypeGuard`` functions which return +``True`` if their input parameter matches their target type. For example, a +function that returns ``TypeGuard[str]`` is assumed to return ``True`` if and +only if it's input parameter is a ``str``. This allows type checkers to narrow +types in this positive case. -This PEP further refines :pep:`647` by allowing type checkers to also narrow types -when a ``TypeGuard`` function returns false. +This PEP further refines :pep:`647` by allowing type checkers to also narrow +types when a ``TypeGuard`` function returns ``False``. Motivation ========== ``TypeGuards`` are used throughout Python libraries to allow a type checker to -narrow the type of something when the ``TypeGuard`` returns true. +narrow the type of something when the ``TypeGuard`` returns ``True``. -However, in the ``else`` clause, :pep:`647` didn't prescribe what the type might -be: +However, in the ``else`` clause, :pep:`647` doesn't prescribe what the type +might be: .. code-block:: python @@ -49,7 +51,7 @@ Specification ============= This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype -of the type of the first input parameter, then the false return case can be +of the type of the first input parameter, then the ``False`` return case can be further narrowed. This changes the example above like so: @@ -68,15 +70,15 @@ This changes the example above like so: Since the ``TypeGuard`` type (or output type) is a subtype of the first input parameter type, a type checker can determine that the only possible type in the ``else`` is the other type in the ``Union``. In this example, it is safe to -assume that if ``is_str`` returns false, then type of the ``val`` argument is an -``int``. +assume that if ``is_str`` returns ``False``, then type of the ``val`` argument +is an ``int``. Unsafe Narrowing --------------------- +---------------- -There are cases where this further type narrowing is not possible though. Here's -an example: +There are cases where this further type narrowing is not possible though. +As an example: .. code-block:: python @@ -89,21 +91,21 @@ an example: else: # Type checker cannot assume list[int] here -Since ``list`` is invariant, it doesn't have any subtypes. ``list[str]`` is not -a subtype of ``list[str | int]``. This means type checkers cannot narrow the -type to ``list[int]`` in the false case. +Since ``list`` is invariant, it doesn't have any subtypes --- ``list[str]`` is +not a subtype of ``list[str | int]``. This means type checkers cannot narrow +the type to ``list[int]`` in the false case. Type checkers should not assume any narrowing in the false case when the ``TypeGuard`` type argument is not a subtype of the first input parameter type. -However, narrowing in the true case is still possible. In the example above, the -type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` -function returns true. +However, narrowing in the true case is still possible. In the example above, +the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` +function returns true. This was established by :pep:`647` and remains true. User error --------------------------- +---------- -The new ``else`` case for a ``TypeGuard`` can be setup incorrectly. Here's an +The new ``else`` case for a ``TypeGuard`` can be set up incorrectly. As an example: .. code-block:: python @@ -118,8 +120,8 @@ example: # Type checker assumes str here A type checker will assume for the else case that the value is ``str``. This is -a change in behavior from :pep:`647` but as that pep stated, there are many ways -a determined or uninformed developer can subvert type safety. +a change in behavior from :pep:`647` but as that PEP stated, there are many +ways a determined or uninformed developer can subvert type safety. A better way to handle this example would be something like so: @@ -140,7 +142,7 @@ A better way to handle this example would be something like so: Backwards Compatibility ======================= -For preexisting code this PEP should require no changes. +For preexisting code, this PEP should require no changes. However, some use cases such as the one below can be simplified: @@ -170,7 +172,7 @@ However, some use cases such as the one below can be simplified: With this proposed change, the code above continues to work but could be -simplified by removing the assertion that x is of type B in the negative case: +simplified by removing the assertion that ``x`` is of type ``B`` in the negative case: .. code-block:: python @@ -195,8 +197,9 @@ simplified by removing the assertion that x is of type B in the negative case: How to Teach This ================= -The belief is that new users will assume this is how ``TypeGuard`` works in the -first place. Meaning this change should make ``TypeGuard`` easier to teach. +We assert that users unfamiliar with ``TypeGuard`` will expect the behavior +outlined in this PEP, therefore making ``TypeGuard`` easier to teach and +explain. Reference Implementation @@ -210,8 +213,8 @@ Rejected Ideas Originally a new ``StrictTypeGuard`` construct was proposed. A ``StrictTypeGuard`` would be similar to to a ``TypeGuard`` except it would -explicitly state that output type was a subtype of the input type. Type checkers -would validate that the output type was a subtype of the input type. +explicitly state that output type was a subtype of the input type. Type +checkers would validate that the output type was a subtype of the input type. See this comment: `StrictTypeGuard proposal `__ From 07af8f7b0dd6c988536e5167a1fb86ab3761de62 Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Fri, 4 Aug 2023 15:26:13 -0700 Subject: [PATCH 29/67] Update pep-0724.rst Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- pep-0724.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index 395a7c74ab8..24d45343c8b 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -218,7 +218,9 @@ Originally a new ``StrictTypeGuard`` construct was proposed. A explicitly state that output type was a subtype of the input type. Type checkers would validate that the output type was a subtype of the input type. -See this comment: `StrictTypeGuard proposal `__ +See this comment: `StrictTypeGuard proposal`__ + +__ https://github.com/python/typing/discussions/1013#discussioncomment-1966238 This was rejected because for most cases it's not necessary. Most people assume the negative case for ``TypeGuard`` anyway, so why not just change the From 81bc56cf9c1635d672e4bfc4c2b417742df798d1 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 15:35:55 -0700 Subject: [PATCH 30/67] More review feedback --- pep-0724.rst | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 24d45343c8b..508c609a2bf 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -12,6 +12,7 @@ Content-Type: text/x-rst Created: 28-Jul-2023 Python-Version: 3.13 Post-History: +Resolution: Abstract @@ -20,7 +21,7 @@ Abstract :pep:`647` introduced the concept of ``TypeGuard`` functions which return ``True`` if their input parameter matches their target type. For example, a function that returns ``TypeGuard[str]`` is assumed to return ``True`` if and -only if it's input parameter is a ``str``. This allows type checkers to narrow +only if its input parameter is a ``str``. This allows type checkers to narrow types in this positive case. This PEP further refines :pep:`647` by allowing type checkers to also narrow @@ -50,9 +51,9 @@ might be: Specification ============= -This PEP proposes that when the type argument of the ``TypeGuard`` is a subtype -of the type of the first input parameter, then the ``False`` return case can be -further narrowed. +This PEP proposes that when the type argument of the ``TypeGuard`` is +consistent[#isconsistent]_ with the type of the first input parameter, then the +``False`` return case can be further narrowed. This changes the example above like so: @@ -67,11 +68,11 @@ This changes the example above like so: else: # Type checkers can assume val is an 'int' in this branch -Since the ``TypeGuard`` type (or output type) is a subtype of the first input -parameter type, a type checker can determine that the only possible type in the -``else`` is the other type in the ``Union``. In this example, it is safe to -assume that if ``is_str`` returns ``False``, then type of the ``val`` argument -is an ``int``. +Since the ``TypeGuard`` type (or output type) is consistent[#isconsistent] with +the first input parameter type, a type checker can determine that the only +possible type in the ``else`` is the other type in the ``Union``. In this +example, it is safe to assume that if ``is_str`` returns ``False``, then type +of the ``val`` argument is an ``int``. Unsafe Narrowing @@ -226,6 +227,11 @@ This was rejected because for most cases it's not necessary. Most people assume the negative case for ``TypeGuard`` anyway, so why not just change the specification to match their assumptions? +Footnotes +========= + +.. [#isconsistent] PEP 483's discussion of ``is-consistent`` + Copyright ========= From 535ab812720a4ce40fdc28c67148558201387bec Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 15:43:25 -0700 Subject: [PATCH 31/67] Fix footnotes --- pep-0724.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 508c609a2bf..64826f749ca 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -52,7 +52,7 @@ Specification ============= This PEP proposes that when the type argument of the ``TypeGuard`` is -consistent[#isconsistent]_ with the type of the first input parameter, then the +consistent [#isconsistent]_ with the type of the first input parameter, then the ``False`` return case can be further narrowed. This changes the example above like so: @@ -68,7 +68,7 @@ This changes the example above like so: else: # Type checkers can assume val is an 'int' in this branch -Since the ``TypeGuard`` type (or output type) is consistent[#isconsistent] with +Since the ``TypeGuard`` type (or output type) is consistent [#isconsistent]_ with the first input parameter type, a type checker can determine that the only possible type in the ``else`` is the other type in the ``Union``. In this example, it is safe to assume that if ``is_str`` returns ``False``, then type @@ -230,7 +230,7 @@ specification to match their assumptions? Footnotes ========= -.. [#isconsistent] PEP 483's discussion of ``is-consistent`` +.. [#isconsistent] :pep:`PEP 483's discussion of is-consistent <483#summary-of-gradual-typing>` Copyright ========= From 7b699481d915f7ed4c44f74b0490019f9bac8f87 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 16:26:11 -0700 Subject: [PATCH 32/67] Update based on Eric's feedback --- pep-0724.rst | 149 +++++++++++++++++---------------------------------- 1 file changed, 48 insertions(+), 101 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 64826f749ca..0a4e4085e71 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -22,7 +22,7 @@ Abstract ``True`` if their input parameter matches their target type. For example, a function that returns ``TypeGuard[str]`` is assumed to return ``True`` if and only if its input parameter is a ``str``. This allows type checkers to narrow -types in this positive case. +types when a ``TypeGuard`` function returns ``True``. This PEP further refines :pep:`647` by allowing type checkers to also narrow types when a ``TypeGuard`` function returns ``False``. @@ -51,15 +51,31 @@ might be: Specification ============= -This PEP proposes that when the type argument of the ``TypeGuard`` is -consistent [#isconsistent]_ with the type of the first input parameter, then the -``False`` return case can be further narrowed. +This PEP proposes some modifications to :pep:`647` in order to allow a type +checker to further narrow what a ``TypeGuard`` is indicating when it returns +``False``. -This changes the example above like so: +* When the output type of a ``TypeGuard`` is consistent [#isconsistent]_ with + the type of its first input parameter, type checkers will apply stricter type + semantics: .. code-block:: python + + # Stricter TypeGuard possible (Kangaroo | Koala is consistent with Animal) + def is_marsupial(val: Animal) -> TypeGuard[Kangaroo | Koala]: + return isinstance(val, Kangaroo | Koala) - def is_str(val: str | int) -> TypeGuard[str]: + # Stricter TypeGuard not possible (list[T] is not consistent with + # list[T | None]) + def has_no_nones(val: list[T | None]) -> TypeGuard[list[T]]: + return None not in val + +* When stricter semantics are possible, the output type for the ``False`` case + can be determined: + +.. code-block:: python + + def is_str(val: str | int) -> TypeGuard[str]: # Stricter mode possible return isinstance(val, str) def func(val: str | int): @@ -68,46 +84,28 @@ This changes the example above like so: else: # Type checkers can assume val is an 'int' in this branch -Since the ``TypeGuard`` type (or output type) is consistent [#isconsistent]_ with -the first input parameter type, a type checker can determine that the only -possible type in the ``else`` is the other type in the ``Union``. In this -example, it is safe to assume that if ``is_str`` returns ``False``, then type -of the ``val`` argument is an ``int``. - - -Unsafe Narrowing ----------------- - -There are cases where this further type narrowing is not possible though. -As an example: +* When the output type of the ``TypeGuard`` is a union, the type checker can + apply additional type narrowing based on the type of the first input + argument, eliminating union elements that are impossible given the input + argument type: .. code-block:: python - def is_str_list(val: list[int | str]) -> TypeGuard[list[str]] - return all(isinstance(x, str) for x in val) + def is_cardinal_direction(val: str) -> TypeGuard[Literal["N", "S", "E", "W"]]: + return val in ("N", "S", "E", "W") - def func(val: list[int | str]): - if is_str_list(val): - # Type checker assumes list[str] here + def func(direction: Literal["NW", "E"]): + if is_cardinal_direction(direction): + # Literal["E"] # The type cannot be "N", "S" or "W" here because of + # argument type else: - # Type checker cannot assume list[int] here - -Since ``list`` is invariant, it doesn't have any subtypes --- ``list[str]`` is -not a subtype of ``list[str | int]``. This means type checkers cannot narrow -the type to ``list[int]`` in the false case. + # Literal["NW"] -Type checkers should not assume any narrowing in the false case when the -``TypeGuard`` type argument is not a subtype of the first input parameter type. - -However, narrowing in the true case is still possible. In the example above, -the type checker can assume the list is a ``list[str]`` if the ``TypeGuard`` -function returns true. This was established by :pep:`647` and remains true. - -User error ----------- +Backwards Compatibility +======================= -The new ``else`` case for a ``TypeGuard`` can be set up incorrectly. As an -example: +The new ``False`` case for a ``TypeGuard`` can lead to incorrect assumptions. +As an example: .. code-block:: python @@ -120,9 +118,9 @@ example: else: # Type checker assumes str here -A type checker will assume for the else case that the value is ``str``. This is -a change in behavior from :pep:`647` but as that PEP stated, there are many -ways a determined or uninformed developer can subvert type safety. +A type checker will assume in the ``False`` case that the value is ``str``. +This is a change in behavior from :pep:`647` but as that PEP stated, there are +many ways a determined or uninformed developer can subvert type safety. A better way to handle this example would be something like so: @@ -139,61 +137,8 @@ A better way to handle this example would be something like so: else: # Type checker assumes str | int here - -Backwards Compatibility -======================= - -For preexisting code, this PEP should require no changes. - -However, some use cases such as the one below can be simplified: - -.. code-block:: python - - class A(): - pass - class B(): - pass - - def is_A(x: A | B) -> TypeGuard[A]: - return is_instance(x, A) - - - def is_B(x: A | B) -> TypeGuard[B]: - return is_instance(x, B) - - - def test(x: A | B): - if is_A(x): - # Do stuff assuming x is an 'A' - return - assert is_B(x) - - # Do stuff assuming x is a 'B' - return - - -With this proposed change, the code above continues to work but could be -simplified by removing the assertion that ``x`` is of type ``B`` in the negative case: - -.. code-block:: python - - class A(): - pass - class B(): - pass - - def is_A(x: A | B) -> TypeGuard[A]: - return is_instance(x, A) - - - def test(x: A | B): - if is_A(x): - # Do stuff assuming x is an 'A' - return - - # Do stuff assuming x is a 'B' - return - +The assumption is that anyone using stricter ``TypeGuard``s will be able to +understand these limitations. How to Teach This ================= @@ -216,16 +161,18 @@ Rejected Ideas Originally a new ``StrictTypeGuard`` construct was proposed. A ``StrictTypeGuard`` would be similar to to a ``TypeGuard`` except it would -explicitly state that output type was a subtype of the input type. Type -checkers would validate that the output type was a subtype of the input type. +explicitly state that output type was consistent [#isconsistent]_ with the +input type. Type checkers would validate that the output type was consistent +[#isconsistent]_ of the input type. See this comment: `StrictTypeGuard proposal`__ __ https://github.com/python/typing/discussions/1013#discussioncomment-1966238 This was rejected because for most cases it's not necessary. Most people assume -the negative case for ``TypeGuard`` anyway, so why not just change the -specification to match their assumptions? +when the ``TypeGuard`` returns ``False``, that the input type has been narrowed +to its other type. Why not just change the specification to match their +assumptions? Footnotes ========= From ddc4ec3183e25b2663080e207cb400edd4b0e08b Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 16:31:55 -0700 Subject: [PATCH 33/67] Fix build warnings --- pep-0724.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 0a4e4085e71..774c5a09e47 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -33,7 +33,7 @@ Motivation ``TypeGuards`` are used throughout Python libraries to allow a type checker to narrow the type of something when the ``TypeGuard`` returns ``True``. -However, in the ``else`` clause, :pep:`647` doesn't prescribe what the type +However, in the ``False`` case, :pep:`647` doesn't prescribe what the type might be: .. code-block:: python @@ -137,7 +137,7 @@ A better way to handle this example would be something like so: else: # Type checker assumes str | int here -The assumption is that anyone using stricter ``TypeGuard``s will be able to +The assumption is that anyone using stricter ``TypeGuards`` will be able to understand these limitations. How to Teach This From dd4bf80d36008606e827bff204450287a8a5bbd5 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 16:32:56 -0700 Subject: [PATCH 34/67] Fixup comment --- pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index 774c5a09e47..ac4e86c28db 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -96,7 +96,7 @@ checker to further narrow what a ``TypeGuard`` is indicating when it returns def func(direction: Literal["NW", "E"]): if is_cardinal_direction(direction): - # Literal["E"] # The type cannot be "N", "S" or "W" here because of + # Literal["E"] The type cannot be "N", "S" or "W" here because of # argument type else: # Literal["NW"] From 5eab93102a0ae3793836e8199f33ccca4cf7a88a Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 17:02:53 -0700 Subject: [PATCH 35/67] Add some more examples --- pep-0724.rst | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/pep-0724.rst b/pep-0724.rst index ac4e86c28db..c65938cf098 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -101,6 +101,52 @@ checker to further narrow what a ``TypeGuard`` is indicating when it returns else: # Literal["NW"] +Additional Examples +=================== + +Any +--- + +``Any`` is consistent [#isconsistent]_ with any other type, which means +stricter semantics can be applied. + +.. code-block:: python + + def is_a(x: Any) -> TypeGuard[A]: # Stricter TypeGuard possible + return isinstance(x, A) + + def test(x: A | B): + if is_a(x): + # x is of type A here + else: + # x is of type B here + +Covariance +---------- + +Covariance is not the same thing as being consistent [#isconsistent]_: + +.. code-block:: python + + def is_str_seq(val: Sequence[str | int]) -> TypeGuard[Sequence[str]]: + return all(isinstance(x, str) for x in val) + + def func3(val: Sequence[str | int]): + if is_str_seq(val): + # val is Sequence[str] + else: + # val is Sequence[str | int] + +Type ``B`` is consistent [#isconsistent]_ with type ``A`` if + +* ``B`` is a subtype of ``A`` +* ``A`` is Any +* ``B`` is Any + +``Sequence[str]`` is not a subtype of ``Sequence[str | int]`` so the special +semantics cannot be applied and only the ``True`` case is narrowed. + + Backwards Compatibility ======================= From c5449b89bcf5bef1bd06f6aa48efa00d5018da1e Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 17:21:22 -0700 Subject: [PATCH 36/67] Rework 'backwards compatibility' --- pep-0724.rst | 45 ++++++++++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index c65938cf098..3498faaa20e 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -146,12 +146,11 @@ Type ``B`` is consistent [#isconsistent]_ with type ``A`` if ``Sequence[str]`` is not a subtype of ``Sequence[str | int]`` so the special semantics cannot be applied and only the ``True`` case is narrowed. +Degenerate case +--------------- -Backwards Compatibility -======================= - -The new ``False`` case for a ``TypeGuard`` can lead to incorrect assumptions. -As an example: +It is possible to break the type checker by making a ``TypeGuard`` that is +incorrect: .. code-block:: python @@ -162,29 +161,37 @@ As an example: if is_positive_int(val): # Type checker assumes int here else: - # Type checker assumes str here + # Type checker assumes str incorrectly here -A type checker will assume in the ``False`` case that the value is ``str``. -This is a change in behavior from :pep:`647` but as that PEP stated, there are -many ways a determined or uninformed developer can subvert type safety. +As :pep:`647 <647#enforcing-strict-narrowing>` stated, users can create invalid +``TypeGuards``. The same thing applies to this PEP. If the user is not careful +the type checker may flag things incorrectly. -A better way to handle this example would be something like so: +Backwards Compatibility +======================= + +The new ``False`` case for a ``TypeGuard`` breaks backwards compatibility. .. code-block:: python - PosInt = NewType('PosInt', int) - - def is_positive_int(val: PosInt | int | str) -> TypeGuard[PosInt]: - return isinstance(val, int) and val > 0 + def is_int(val: int | str) -> TypeGuard[int]: + return isinstance(val, int) def func(val: int | str): - if is_positive_int(val): - # Type checker assumes PosInt here + if is_int(val): + # Type checker assumes int here else: - # Type checker assumes str | int here + # Type checker assumes str here + +A type checker will assume in the ``False`` case that the value is ``str``. + +This is a change in behavior from :pep:`647`. + +TBD: Not sure what the resolution of this should be yet. I want to say +something like - Code that already exists would not be harmed by this change as +existing code would have more checks to validate the type on the else case. For +new code, the assumption is that users will expect this behavior anyway. -The assumption is that anyone using stricter ``TypeGuards`` will be able to -understand these limitations. How to Teach This ================= From 6ff5be96442e9cb156fb237d477783f318fb473e Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 4 Aug 2023 17:22:59 -0700 Subject: [PATCH 37/67] Grammar --- pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index 3498faaa20e..e508cd14714 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -216,7 +216,7 @@ Originally a new ``StrictTypeGuard`` construct was proposed. A ``StrictTypeGuard`` would be similar to to a ``TypeGuard`` except it would explicitly state that output type was consistent [#isconsistent]_ with the input type. Type checkers would validate that the output type was consistent -[#isconsistent]_ of the input type. +[#isconsistent]_ with the input type. See this comment: `StrictTypeGuard proposal`__ From 487a47e6149b8bd113927f03df46e481d353328d Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Date: Sat, 5 Aug 2023 02:07:46 +0100 Subject: [PATCH 38/67] Remove ``Resolution`` --- pep-0724.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index e508cd14714..801776537f7 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -12,7 +12,6 @@ Content-Type: text/x-rst Created: 28-Jul-2023 Python-Version: 3.13 Post-History: -Resolution: Abstract From 4f9753ad338030fc449c72fee10bbf375d97a73d Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Mon, 7 Aug 2023 09:22:15 -0700 Subject: [PATCH 39/67] Update pep-0724.rst Co-authored-by: Hugo van Kemenade --- pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index 801776537f7..a16f46042ef 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -29,7 +29,7 @@ types when a ``TypeGuard`` function returns ``False``. Motivation ========== -``TypeGuards`` are used throughout Python libraries to allow a type checker to +``TypeGuard``\ s are used throughout Python libraries to allow a type checker to narrow the type of something when the ``TypeGuard`` returns ``True``. However, in the ``False`` case, :pep:`647` doesn't prescribe what the type From 3c37ec34ae5d1afda31c755f88c09db5e1e4d49d Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Mon, 7 Aug 2023 09:22:22 -0700 Subject: [PATCH 40/67] Update pep-0724.rst Co-authored-by: Hugo van Kemenade --- pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index a16f46042ef..ebf1b31b794 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -163,7 +163,7 @@ incorrect: # Type checker assumes str incorrectly here As :pep:`647 <647#enforcing-strict-narrowing>` stated, users can create invalid -``TypeGuards``. The same thing applies to this PEP. If the user is not careful +``TypeGuard``\ s. The same thing applies to this PEP. If the user is not careful the type checker may flag things incorrectly. Backwards Compatibility From b90df2c6afbae9a95fc0a7ba03e5ec3d37ffc104 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Mon, 7 Aug 2023 09:42:39 -0700 Subject: [PATCH 41/67] Fix column width --- pep-0724.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index ebf1b31b794..f34de123ebb 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -29,8 +29,8 @@ types when a ``TypeGuard`` function returns ``False``. Motivation ========== -``TypeGuard``\ s are used throughout Python libraries to allow a type checker to -narrow the type of something when the ``TypeGuard`` returns ``True``. +``TypeGuard``\ s are used throughout Python libraries to allow a type checker +to narrow the type of something when the ``TypeGuard`` returns ``True``. However, in the ``False`` case, :pep:`647` doesn't prescribe what the type might be: @@ -163,8 +163,8 @@ incorrect: # Type checker assumes str incorrectly here As :pep:`647 <647#enforcing-strict-narrowing>` stated, users can create invalid -``TypeGuard``\ s. The same thing applies to this PEP. If the user is not careful -the type checker may flag things incorrectly. +``TypeGuard``\ s. The same thing applies to this PEP. If the user is not +careful the type checker may flag things incorrectly. Backwards Compatibility ======================= From 3263b631d34654e30ee6219c193d79eac2e2209b Mon Sep 17 00:00:00 2001 From: rchiodo Date: Tue, 8 Aug 2023 09:27:31 -0700 Subject: [PATCH 42/67] Another try for backwards compatibility --- pep-0724.rst | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index f34de123ebb..a043204eb4b 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -186,10 +186,26 @@ A type checker will assume in the ``False`` case that the value is ``str``. This is a change in behavior from :pep:`647`. -TBD: Not sure what the resolution of this should be yet. I want to say -something like - Code that already exists would not be harmed by this change as -existing code would have more checks to validate the type on the else case. For -new code, the assumption is that users will expect this behavior anyway. +Although this is different behavior, existing code would likely have further +checks for a ``str`` in the ``False`` case. Since the previous behavior was to +assume ``str | int``, existing code would have had to further refine the type +to get a type checker to behave the expected way. + +Something like so: + +.. code-block:: python + + def is_int(val: int | str) -> TypeGuard[int]: + return isinstance(val, int) + + def is_str(val: int | str) -> TypeGuard[int]: + return isinstance(val, int) + + def func(val: int | str): + if is_int(val): + # Code does something with int here + else is_str(val): # This check would have been necessary before + # Code does something with str here How to Teach This From dcf3d4940135d50b666442c235b7bad6410e8e20 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Tue, 8 Aug 2023 09:29:34 -0700 Subject: [PATCH 43/67] another blurb about backwards compatibility --- pep-0724.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/pep-0724.rst b/pep-0724.rst index a043204eb4b..5b0a91e859c 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -208,6 +208,10 @@ Something like so: # Code does something with str here +This means existing code would continue to work with the ``False`` narrowing. +Meaning this change could be considered backwards compatible because it's not +actually changing any runtime behavior. + How to Teach This ================= From a580f4ace07f565043df7ba3bb3b9bebc450e1fd Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 25 Aug 2023 16:36:06 -0700 Subject: [PATCH 44/67] Add mypy primer test --- pep-0724.rst | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 5b0a91e859c..d39c6600a08 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -208,9 +208,20 @@ Something like so: # Code does something with str here -This means existing code would continue to work with the ``False`` narrowing. -Meaning this change could be considered backwards compatible because it's not -actually changing any runtime behavior. +This means existing code should continue to work with the ``False`` narrowing. + +As a proof of this concept, we ran the `experimental Pyright changes`__ +against `mypy primer`__ to see if there were any differences in the output. +Changing ``TypeGuard`` to be more strict had a small effect on the ``mypy +primer``. + +__ https://github.com/microsoft/pyright/pull/5813 +__ https://github.com/hauntsaninja/mypy_primer + +TBD: Maybe there's some bugs in the implementation. We thought there +would be no differences but that doesn't seem to be the case. Checking +with Eric. + How to Teach This ================= From 95d4616fb684e0c3856a051eded6de0d809c985d Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 25 Aug 2023 16:58:21 -0700 Subject: [PATCH 45/67] Add another rejected idea --- pep-0724.rst | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/pep-0724.rst b/pep-0724.rst index d39c6600a08..0e7e93bfa90 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -242,6 +242,9 @@ __ https://github.com/microsoft/pyright/commit/9a5af798d726bd0612cebee7223676c39 Rejected Ideas ============== +StrictTypeGuard +--------------- + Originally a new ``StrictTypeGuard`` construct was proposed. A ``StrictTypeGuard`` would be similar to to a ``TypeGuard`` except it would explicitly state that output type was consistent [#isconsistent]_ with the @@ -257,6 +260,28 @@ when the ``TypeGuard`` returns ``False``, that the input type has been narrowed to its other type. Why not just change the specification to match their assumptions? +TypeGuard with a second output type +----------------------------------- + +Another idea was also proposed where a ``TypeGuard`` could have a second output +type. Something like so: + +.. code-block:: python + + def is_int(val: int | str) -> TypeGuard[int, str]: + return isinstance(val, int) + + +The second output type tells a type checker what type is returned in the ``False`` +case. It was originally proposed `here.`__ + +__ https://github.com/python/typing/issues/996 + +This idea was rejected because of the negative feedback it received. +Specifically this `thread.`__ + +__ https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL + Footnotes ========= From b267456de05fd0a4d308a46299bc916c477c45ca Mon Sep 17 00:00:00 2001 From: rchiodo Date: Mon, 28 Aug 2023 08:55:08 -0700 Subject: [PATCH 46/67] Fix typo --- pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index 0e7e93bfa90..5ce119d329f 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -246,7 +246,7 @@ StrictTypeGuard --------------- Originally a new ``StrictTypeGuard`` construct was proposed. A -``StrictTypeGuard`` would be similar to to a ``TypeGuard`` except it would +``StrictTypeGuard`` would be similar to a ``TypeGuard`` except it would explicitly state that output type was consistent [#isconsistent]_ with the input type. Type checkers would validate that the output type was consistent [#isconsistent]_ with the input type. From 6b246f23e987f10d3b92077f650bf93bfa8bf758 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Wed, 30 Aug 2023 15:14:33 -0700 Subject: [PATCH 47/67] Update based on discussions with Eric --- pep-0724.rst | 64 +++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 51 insertions(+), 13 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 5ce119d329f..ff7eb90ab44 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -50,13 +50,38 @@ might be: Specification ============= +Given a ``TypeGuard`` and a calling function like so: + +.. code-block:: python + + def guard(x: P) -> TypeGuard[R]: ... + + def func1(val: A): + if guard(val): + reveal_type(val) # NP + else: + reveal_type(val) # NN + +Each use of a ``TypeGuard`` involves five types: + +* P = ``TypeGuard`` parameter type + +* R = ``TypeGuard`` return type + +* A = Argument type (pre-narrowed) + +* NP = Narrowed type (positive) + +* NN = Narrowed type (negative) + + This PEP proposes some modifications to :pep:`647` in order to allow a type -checker to further narrow what a ``TypeGuard`` is indicating when it returns -``False``. +checker to further narrow what a ``TypeGuard`` returns when the ``R`` type is +consistent [#isconsistent]_ with the ``P`` type. -* When the output type of a ``TypeGuard`` is consistent [#isconsistent]_ with - the type of its first input parameter, type checkers will apply stricter type - semantics: +* When the output type ``R`` of a ``TypeGuard`` is consistent [#isconsistent]_ + with the type of its first input parameter (``P``), type checkers will apply + stricter type semantics: .. code-block:: python @@ -69,8 +94,8 @@ checker to further narrow what a ``TypeGuard`` is indicating when it returns def has_no_nones(val: list[T | None]) -> TypeGuard[list[T]]: return None not in val -* When stricter semantics are possible, the output type for the ``False`` case - can be determined: +* When stricter semantics are possible, the output type for the ``False`` or + ``NN`` case can be determined: .. code-block:: python @@ -100,6 +125,22 @@ checker to further narrow what a ``TypeGuard`` is indicating when it returns else: # Literal["NW"] + +This logic can be summed up in a table: + +============ ======================= =================== +|space| Non strict TypeGuard Strict TypeGuard +============ ======================= =================== +Applies when R not consistent with P R consistent with P +NP is .. R A & R +NN is .. A A & ~R +============ ======================= =================== + +The definition of ``A & R`` and ``A & ~R`` would be left up to +the type checker itself as these concepts don't yet exist in Python typing. + +.. |space| unicode:: 0x2008 + Additional Examples =================== @@ -213,15 +254,12 @@ This means existing code should continue to work with the ``False`` narrowing. As a proof of this concept, we ran the `experimental Pyright changes`__ against `mypy primer`__ to see if there were any differences in the output. Changing ``TypeGuard`` to be more strict had a small effect on the ``mypy -primer``. +primer``. Mostly indicating that some ``# type: ignore`` comments were no longer +necessary. -__ https://github.com/microsoft/pyright/pull/5813 +__ https://github.com/microsoft/pyright/pull/5832 __ https://github.com/hauntsaninja/mypy_primer -TBD: Maybe there's some bugs in the implementation. We thought there -would be no differences but that doesn't seem to be the case. Checking -with Eric. - How to Teach This ================= From 0e24a6fc81525459193d5c38c228cc960cbafe36 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Thu, 31 Aug 2023 09:19:37 -0700 Subject: [PATCH 48/67] Remove unnecessary lines and incorrect statement about covariance --- pep-0724.rst | 29 ----------------------------- 1 file changed, 29 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index ff7eb90ab44..334fec95436 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -65,13 +65,9 @@ Given a ``TypeGuard`` and a calling function like so: Each use of a ``TypeGuard`` involves five types: * P = ``TypeGuard`` parameter type - * R = ``TypeGuard`` return type - * A = Argument type (pre-narrowed) - * NP = Narrowed type (positive) - * NN = Narrowed type (negative) @@ -161,31 +157,6 @@ stricter semantics can be applied. else: # x is of type B here -Covariance ----------- - -Covariance is not the same thing as being consistent [#isconsistent]_: - -.. code-block:: python - - def is_str_seq(val: Sequence[str | int]) -> TypeGuard[Sequence[str]]: - return all(isinstance(x, str) for x in val) - - def func3(val: Sequence[str | int]): - if is_str_seq(val): - # val is Sequence[str] - else: - # val is Sequence[str | int] - -Type ``B`` is consistent [#isconsistent]_ with type ``A`` if - -* ``B`` is a subtype of ``A`` -* ``A`` is Any -* ``B`` is Any - -``Sequence[str]`` is not a subtype of ``Sequence[str | int]`` so the special -semantics cannot be applied and only the ``True`` case is narrowed. - Degenerate case --------------- From 5ff0408482b6a0dd283ada2300b33bc6beafacce Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 1 Sep 2023 09:40:25 -0700 Subject: [PATCH 49/67] Use 'math' to specify logic Reference intersection and negation docs --- pep-0724.rst | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 334fec95436..55d7f476830 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -128,12 +128,16 @@ This logic can be summed up in a table: |space| Non strict TypeGuard Strict TypeGuard ============ ======================= =================== Applies when R not consistent with P R consistent with P -NP is .. R A & R -NN is .. A A & ~R +NP is .. :math:`R` :math:`A \land R` +NN is .. :math:`A` :math:`A \land \neg{R}` ============ ======================= =================== -The definition of ``A & R`` and ``A & ~R`` would be left up to -the type checker itself as these concepts don't yet exist in Python typing. +Intersection of types is being discussed `here`__ and will likely be specified +in a future PEP. Negation of types may end up being implemented by each type +checker as the negation of gradual types is `not well defined`__. + +__ https://github.com/CarliJoy/intersection_examples/blob/main/specification.rst +__ https://github.com/python/typing/issues/801#issuecomment-1679883519 .. |space| unicode:: 0x2008 From d8ea2ccc7396ebfe40f1f356b654b411ee6552d6 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 1 Sep 2023 10:16:06 -0700 Subject: [PATCH 50/67] Don't reference intersections as a future PEP --- pep-0724.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 55d7f476830..faa2b352fa3 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -132,12 +132,12 @@ NP is .. :math:`R` :math:`A \land R` NN is .. :math:`A` :math:`A \land \neg{R}` ============ ======================= =================== -Intersection of types is being discussed `here`__ and will likely be specified -in a future PEP. Negation of types may end up being implemented by each type -checker as the negation of gradual types is `not well defined`__. +Intersection of types and type negation are not defined at the moment and are +therefore left up to the type checker to decide on how to implement. Future +`extensions`__ to the type system may change this behavior though. + __ https://github.com/CarliJoy/intersection_examples/blob/main/specification.rst -__ https://github.com/python/typing/issues/801#issuecomment-1679883519 .. |space| unicode:: 0x2008 From 671df1fdcb6c28892ca6f0031c83bef22ac51fbc Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 1 Sep 2023 10:16:24 -0700 Subject: [PATCH 51/67] Extra line --- pep-0724.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/pep-0724.rst b/pep-0724.rst index faa2b352fa3..736c6e9ebef 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -136,7 +136,6 @@ Intersection of types and type negation are not defined at the moment and are therefore left up to the type checker to decide on how to implement. Future `extensions`__ to the type system may change this behavior though. - __ https://github.com/CarliJoy/intersection_examples/blob/main/specification.rst .. |space| unicode:: 0x2008 From d3048de2d4c85f3b73dc7b29611f3b154cad27ad Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 1 Sep 2023 10:18:58 -0700 Subject: [PATCH 52/67] Improve wording --- pep-0724.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 736c6e9ebef..d19ec9202cf 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -132,9 +132,9 @@ NP is .. :math:`R` :math:`A \land R` NN is .. :math:`A` :math:`A \land \neg{R}` ============ ======================= =================== -Intersection of types and type negation are not defined at the moment and are -therefore left up to the type checker to decide on how to implement. Future -`extensions`__ to the type system may change this behavior though. +Intersection of types and type negation are not defined and are left up to the +type checker to decide on how to implement. Future `extensions`__ to the type +system may change this behavior though. __ https://github.com/CarliJoy/intersection_examples/blob/main/specification.rst From 4a7039eae7c35102f6434c3bbc151bd89fddbc08 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Tue, 5 Sep 2023 10:23:54 -0700 Subject: [PATCH 53/67] Move degenerate case --- pep-0724.rst | 46 ++++++++++++++++++++++------------------------ 1 file changed, 22 insertions(+), 24 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index d19ec9202cf..1f3d0cace03 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -133,10 +133,8 @@ NN is .. :math:`A` :math:`A \land \neg{R}` ============ ======================= =================== Intersection of types and type negation are not defined and are left up to the -type checker to decide on how to implement. Future `extensions`__ to the type -system may change this behavior though. - -__ https://github.com/CarliJoy/intersection_examples/blob/main/specification.rst +type checker to decide on how to implement. Future extensions to the type +system may define this behavior though. .. |space| unicode:: 0x2008 @@ -160,26 +158,6 @@ stricter semantics can be applied. else: # x is of type B here -Degenerate case ---------------- - -It is possible to break the type checker by making a ``TypeGuard`` that is -incorrect: - -.. code-block:: python - - def is_positive_int(val: int | str) -> TypeGuard[int]: - return isinstance(val, int) and val > 0 - - def func(val: int | str): - if is_positive_int(val): - # Type checker assumes int here - else: - # Type checker assumes str incorrectly here - -As :pep:`647 <647#enforcing-strict-narrowing>` stated, users can create invalid -``TypeGuard``\ s. The same thing applies to this PEP. If the user is not -careful the type checker may flag things incorrectly. Backwards Compatibility ======================= @@ -234,6 +212,26 @@ necessary. __ https://github.com/microsoft/pyright/pull/5832 __ https://github.com/hauntsaninja/mypy_primer +Breaking change +--------------- + +It is possible to create a broken ``TypeGuard`` now: + +.. code-block:: python + + def is_positive_int(val: int | str) -> TypeGuard[int]: + return isinstance(val, int) and val > 0 + + def func(val: int | str): + if is_positive_int(val): + # Type checker assumes int here + else: + # Type checker now assumes str incorrectly here + +This case is now incorrect. However, we're confident that this is not a +real world scenario. The mypy primer run didn't show any similar uses cases. +Dynamic ``TypeGuard`` s are not a common pattern. + How to Teach This ================= From e3fd5582234ff8d2c856ff1d0df8f8461490d060 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Tue, 5 Sep 2023 10:31:53 -0700 Subject: [PATCH 54/67] Some formatting --- pep-0724.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index 1f3d0cace03..c2b5a7e6b54 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -29,7 +29,7 @@ types when a ``TypeGuard`` function returns ``False``. Motivation ========== -``TypeGuard``\ s are used throughout Python libraries to allow a type checker +``TypeGuard``\s are used throughout Python libraries to allow a type checker to narrow the type of something when the ``TypeGuard`` returns ``True``. However, in the ``False`` case, :pep:`647` doesn't prescribe what the type @@ -230,7 +230,7 @@ It is possible to create a broken ``TypeGuard`` now: This case is now incorrect. However, we're confident that this is not a real world scenario. The mypy primer run didn't show any similar uses cases. -Dynamic ``TypeGuard`` s are not a common pattern. +Dynamic ``TypeGuard``\s are not a common pattern. How to Teach This From e5f47490b5f90027ba4a4cbf76792934892b440c Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 7 Sep 2023 17:28:42 -0700 Subject: [PATCH 55/67] Update pep-0724.rst Fix lint --- pep-0724.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0724.rst b/pep-0724.rst index c2b5a7e6b54..997f1c54127 100644 --- a/pep-0724.rst +++ b/pep-0724.rst @@ -1,7 +1,7 @@ PEP: 724 Title: Stricter Type Guards -Author: Rich Chiodo , - Eric Traut , +Author: Rich Chiodo , + Eric Traut , Erik De Bonte Sponsor: Jelle Zijlstra Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ From 44a31ae4e892aa805cef8bbf2e42436a81f857a0 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Sun, 10 Sep 2023 00:42:28 +0100 Subject: [PATCH 56/67] Move to ``peps/`` --- pep-0724.rst => peps/pep-0724.rst | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename pep-0724.rst => peps/pep-0724.rst (100%) diff --git a/pep-0724.rst b/peps/pep-0724.rst similarity index 100% rename from pep-0724.rst rename to peps/pep-0724.rst From 93b1503b35fe59dec0dc71aee8f54fb6ddff4525 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Sun, 10 Sep 2023 00:58:19 +0100 Subject: [PATCH 57/67] Whitespace --- peps/pep-0724.rst | 73 ++++++++++++++++++++++++----------------------- 1 file changed, 37 insertions(+), 36 deletions(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index 997f1c54127..216c1b3f48d 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -11,7 +11,7 @@ Topic: Typing Content-Type: text/x-rst Created: 28-Jul-2023 Python-Version: 3.13 -Post-History: +Post-History: Abstract @@ -26,6 +26,7 @@ types when a ``TypeGuard`` function returns ``True``. This PEP further refines :pep:`647` by allowing type checkers to also narrow types when a ``TypeGuard`` function returns ``False``. + Motivation ========== @@ -44,7 +45,7 @@ might be: if is_str(val): # Type checkers can assume val is a 'str' in this branch else: - # Type here is not narrowed. It is still 'str | int' + # Type here is not narrowed. It is still 'str | int' Specification @@ -58,9 +59,9 @@ Given a ``TypeGuard`` and a calling function like so: def func1(val: A): if guard(val): - reveal_type(val) # NP + reveal_type(val) # NP else: - reveal_type(val) # NN + reveal_type(val) # NN Each use of a ``TypeGuard`` involves five types: @@ -70,7 +71,6 @@ Each use of a ``TypeGuard`` involves five types: * NP = Narrowed type (positive) * NN = Narrowed type (negative) - This PEP proposes some modifications to :pep:`647` in order to allow a type checker to further narrow what a ``TypeGuard`` returns when the ``R`` type is consistent [#isconsistent]_ with the ``P`` type. @@ -79,53 +79,53 @@ consistent [#isconsistent]_ with the ``P`` type. with the type of its first input parameter (``P``), type checkers will apply stricter type semantics: -.. code-block:: python - - # Stricter TypeGuard possible (Kangaroo | Koala is consistent with Animal) - def is_marsupial(val: Animal) -> TypeGuard[Kangaroo | Koala]: - return isinstance(val, Kangaroo | Koala) + .. code-block:: python + + # Stricter TypeGuard possible (Kangaroo | Koala is consistent with Animal) + def is_marsupial(val: Animal) -> TypeGuard[Kangaroo | Koala]: + return isinstance(val, Kangaroo | Koala) - # Stricter TypeGuard not possible (list[T] is not consistent with - # list[T | None]) - def has_no_nones(val: list[T | None]) -> TypeGuard[list[T]]: - return None not in val + # Stricter TypeGuard not possible + # (list[T] is not consistent with list[T | None]) + def has_no_nones(val: list[T | None]) -> TypeGuard[list[T]]: + return None not in val * When stricter semantics are possible, the output type for the ``False`` or ``NN`` case can be determined: -.. code-block:: python + .. code-block:: python - def is_str(val: str | int) -> TypeGuard[str]: # Stricter mode possible - return isinstance(val, str) + def is_str(val: str | int) -> TypeGuard[str]: # Stricter mode possible + return isinstance(val, str) - def func(val: str | int): - if is_str(val): - # Type checkers can assume val is a 'str' in this branch - else: - # Type checkers can assume val is an 'int' in this branch + def func(val: str | int): + if is_str(val): + # Type checkers can assume val is a 'str' in this branch + else: + # Type checkers can assume val is an 'int' in this branch * When the output type of the ``TypeGuard`` is a union, the type checker can apply additional type narrowing based on the type of the first input argument, eliminating union elements that are impossible given the input argument type: -.. code-block:: python + .. code-block:: python - def is_cardinal_direction(val: str) -> TypeGuard[Literal["N", "S", "E", "W"]]: - return val in ("N", "S", "E", "W") + def is_cardinal_direction(val: str) -> TypeGuard[Literal["N", "S", "E", "W"]]: + return val in ("N", "S", "E", "W") - def func(direction: Literal["NW", "E"]): - if is_cardinal_direction(direction): - # Literal["E"] The type cannot be "N", "S" or "W" here because of - # argument type - else: - # Literal["NW"] + def func(direction: Literal["NW", "E"]): + if is_cardinal_direction(direction): + # Literal["E"] The type cannot be "N", "S" or "W" here because of + # argument type + else: + # Literal["NW"] This logic can be summed up in a table: ============ ======================= =================== -|space| Non strict TypeGuard Strict TypeGuard +\ Non strict TypeGuard Strict TypeGuard ============ ======================= =================== Applies when R not consistent with P R consistent with P NP is .. :math:`R` :math:`A \land R` @@ -136,7 +136,6 @@ Intersection of types and type negation are not defined and are left up to the type checker to decide on how to implement. Future extensions to the type system may define this behavior though. -.. |space| unicode:: 0x2008 Additional Examples =================== @@ -145,11 +144,11 @@ Any --- ``Any`` is consistent [#isconsistent]_ with any other type, which means -stricter semantics can be applied. +stricter semantics can be applied. .. code-block:: python - def is_a(x: Any) -> TypeGuard[A]: # Stricter TypeGuard possible + def is_a(x: Any) -> TypeGuard[A]: # Stricter TypeGuard possible return isinstance(x, A) def test(x: A | B): @@ -197,7 +196,7 @@ Something like so: def func(val: int | str): if is_int(val): # Code does something with int here - else is_str(val): # This check would have been necessary before + else is_str(val): # This check would have been necessary before # Code does something with str here @@ -292,11 +291,13 @@ Specifically this `thread.`__ __ https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL + Footnotes ========= .. [#isconsistent] :pep:`PEP 483's discussion of is-consistent <483#summary-of-gradual-typing>` + Copyright ========= From 499639f371063fc2c474549f53822511cf0e6f59 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Sun, 10 Sep 2023 00:58:29 +0100 Subject: [PATCH 58/67] Add trailing comma --- peps/pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index 216c1b3f48d..2bc56c2041b 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -2,7 +2,7 @@ PEP: 724 Title: Stricter Type Guards Author: Rich Chiodo , Eric Traut , - Erik De Bonte + Erik De Bonte , Sponsor: Jelle Zijlstra Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ Status: Draft From b181ef9417819b12c927c3ec0fc43cb2cc3a4758 Mon Sep 17 00:00:00 2001 From: Adam Turner <9087854+aa-turner@users.noreply.github.com> Date: Sun, 10 Sep 2023 00:58:43 +0100 Subject: [PATCH 59/67] Copyright formatting --- peps/pep-0724.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index 2bc56c2041b..e4b74f71df4 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -301,5 +301,5 @@ Footnotes Copyright ========= -This document is placed in the public domain or under the CC0-1.0-Universal -license, whichever is more permissive. +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. From 5963e4561fe9d94594685673d7fc7ff8fb62e3fd Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Mon, 11 Sep 2023 09:29:42 -0700 Subject: [PATCH 60/67] Update peps/pep-0724.rst Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- peps/pep-0724.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index e4b74f71df4..b834bbfc3ed 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -204,8 +204,8 @@ This means existing code should continue to work with the ``False`` narrowing. As a proof of this concept, we ran the `experimental Pyright changes`__ against `mypy primer`__ to see if there were any differences in the output. -Changing ``TypeGuard`` to be more strict had a small effect on the ``mypy -primer``. Mostly indicating that some ``# type: ignore`` comments were no longer +Changing ``TypeGuard`` to be more strict had a small effect, +mostly indicating that some ``# type: ignore`` comments were no longer necessary. __ https://github.com/microsoft/pyright/pull/5832 From 867c54102ae748b6ace0c7ddb6f43ef194691f74 Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Mon, 11 Sep 2023 09:34:36 -0700 Subject: [PATCH 61/67] Update peps/pep-0724.rst Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- peps/pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index b834bbfc3ed..53992bd67d0 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -282,7 +282,7 @@ type. Something like so: The second output type tells a type checker what type is returned in the ``False`` -case. It was originally proposed `here.`__ +case. It was originally proposed `here`__. __ https://github.com/python/typing/issues/996 From 2a2336a60f4c441c9d7b3a687889c36ff43b4be4 Mon Sep 17 00:00:00 2001 From: Rich Chiodo Date: Mon, 11 Sep 2023 09:34:58 -0700 Subject: [PATCH 62/67] Update peps/pep-0724.rst Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> --- peps/pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index 53992bd67d0..c25dbbd2710 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -287,7 +287,7 @@ case. It was originally proposed `here`__. __ https://github.com/python/typing/issues/996 This idea was rejected because of the negative feedback it received. -Specifically this `thread.`__ +Specifically this `thread`__. __ https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL From dd5fdee1fee11f94c7b0450bd9eb01538df4de54 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Mon, 11 Sep 2023 09:37:19 -0700 Subject: [PATCH 63/67] More review feedback --- peps/pep-0724.rst | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index c25dbbd2710..53a0caa70ff 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -4,14 +4,14 @@ Author: Rich Chiodo , Eric Traut , Erik De Bonte , Sponsor: Jelle Zijlstra -Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ +Discussions-To: Status: Draft Type: Standards Track Topic: Typing Content-Type: text/x-rst Created: 28-Jul-2023 Python-Version: 3.13 -Post-History: +Post-History: https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ Abstract @@ -140,9 +140,6 @@ system may define this behavior though. Additional Examples =================== -Any ---- - ``Any`` is consistent [#isconsistent]_ with any other type, which means stricter semantics can be applied. From 92768e2255c7d8b50c23562baaea6052c8355d1a Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 15 Sep 2023 16:00:07 -0700 Subject: [PATCH 64/67] Updates from Eric Traut --- peps/pep-0724.rst | 292 +++++++++++++++++++++++++--------------------- 1 file changed, 161 insertions(+), 131 deletions(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index 53a0caa70ff..14135b17da3 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -17,45 +17,87 @@ Post-History: https://mail.python.org/archives/list/typing-sig@python.org/thread Abstract ======== -:pep:`647` introduced the concept of ``TypeGuard`` functions which return -``True`` if their input parameter matches their target type. For example, a -function that returns ``TypeGuard[str]`` is assumed to return ``True`` if and -only if its input parameter is a ``str``. This allows type checkers to narrow -types when a ``TypeGuard`` function returns ``True``. +:pep:`647` introduced the concept of a user-defined type guard function which +returns ``True`` if the type of the expression passed to its first parameter +matches its return ``TypeGuard`` type. For example, a function that has a +return type of ``TypeGuard[str]`` is assumed to return ``True`` if and only if +the type of the expression passed to its first input parameter is a ``str``. +This allows type checkers to narrow types when a user-defined type guard +function returns ``True``. -This PEP further refines :pep:`647` by allowing type checkers to also narrow -types when a ``TypeGuard`` function returns ``False``. +This PEP refines the ``TypeGuard`` mechanism introduced in :pep:`647`. It +allows type checkers to narrow types when a user-defined type guard function +returns ``False``. It also allows type checkers to apply additional (more +precise) type narrowing under certain circumstances when the type guard +function returns ``True``. Motivation ========== -``TypeGuard``\s are used throughout Python libraries to allow a type checker -to narrow the type of something when the ``TypeGuard`` returns ``True``. +User-defined type guard functions enable a type checker to narrow the type of +an expression when it is passed as an argument to the type guard function. The +``TypeGuard`` mechanism introduced in :pep:`647` is flexible, but this +flexibility imposes some limitations that developers have found inconvenient +for some uses. -However, in the ``False`` case, :pep:`647` doesn't prescribe what the type -might be: +Limitation 1: Type checkers are not allowed to narrow a type in the case where +the type guard function returns ``False``. This means the type is not narrowed +in the negative ("else") clause. + +Limitation 2: Type checkers must use the ``TypeGuard`` return type if the type +guard function returns ``True`` regardless of whether additional narrowing can +be applied based on knowledge of the pre-narrowed type. + +The following code sample demonstrates both of these limitations. .. code-block:: python - def is_str(val: str | int) -> TypeGuard[str]: - return isinstance(val, str) + def is_iterable(val: object) -> TypeGuard[Iterable[Any]]: + return isinstance(val, Iterable) - def func(val: str | int): - if is_str(val): - # Type checkers can assume val is a 'str' in this branch + def func(val: int | list[int]): + if is_iterable(val): + # The type is narrowed to 'Iterable[Any]' as dictated by + # the TypeGuard return type + reveal_type(val) # Iterable[Any] + else: + # The type is not narrowed in the "False" case + reveal_type(val) # int | list[int] + + # If "isinstance" is used in place of the user-defined type guard + # function, the results differ because type checkers apply additional + # logic for "isinstance" + + if isinstance(val, Iterable): + # Type is narrowed to "list[int]" because this is + # a narrower (more precise) type than "Iterable[Any]" + reveal_type(val) # list[int] else: - # Type here is not narrowed. It is still 'str | int' + # Type is narrowed to "int" because the logic eliminates + # "list[int]" from the original union + reveal_type(val) # int + + +:pep:`647` imposed these limitations so it could support use cases where the +return ``TypeGuard`` type was not a subtype of the input type. Refer to +:pep:`647` for examples. Specification ============= -Given a ``TypeGuard`` and a calling function like so: +The use of a user-defined type guard function involves five types: + +* I = ``TypeGuard`` input type +* R = ``TypeGuard`` return type +* A = Type of argument passed to type guard function (pre-narrowed) +* NP = Narrowed type (positive) +* NN = Narrowed type (negative) .. code-block:: python - def guard(x: P) -> TypeGuard[R]: ... + def guard(x: I) -> TypeGuard[R]: ... def func1(val: A): if guard(val): @@ -63,78 +105,71 @@ Given a ``TypeGuard`` and a calling function like so: else: reveal_type(val) # NN -Each use of a ``TypeGuard`` involves five types: - -* P = ``TypeGuard`` parameter type -* R = ``TypeGuard`` return type -* A = Argument type (pre-narrowed) -* NP = Narrowed type (positive) -* NN = Narrowed type (negative) - -This PEP proposes some modifications to :pep:`647` in order to allow a type -checker to further narrow what a ``TypeGuard`` returns when the ``R`` type is -consistent [#isconsistent]_ with the ``P`` type. -* When the output type ``R`` of a ``TypeGuard`` is consistent [#isconsistent]_ - with the type of its first input parameter (``P``), type checkers will apply - stricter type semantics: +This PEP proposes some modifications to :pep:`647` to address the limitations +discussed above. These limitations are safe to eliminate only when a specific +condition is met. In particular, when the output type ``R`` of a user-defined +type guard function is consistent [#isconsistent]_ with the type of its first +input parameter (``I``), type checkers should apply stricter type guard +semantics. .. code-block:: python - # Stricter TypeGuard possible (Kangaroo | Koala is consistent with Animal) + # Stricter type guard semantics are used in this case because + # "Kangaroo | Koala" is consistent with "Animal" def is_marsupial(val: Animal) -> TypeGuard[Kangaroo | Koala]: return isinstance(val, Kangaroo | Koala) - # Stricter TypeGuard not possible - # (list[T] is not consistent with list[T | None]) + # Stricter type guard semantics are not used in this case because + # "list[T]"" is not consistent with "list[T | None]" def has_no_nones(val: list[T | None]) -> TypeGuard[list[T]]: return None not in val -* When stricter semantics are possible, the output type for the ``False`` or - ``NN`` case can be determined: +When stricter type guard semantics are applied, the application of a +user-defined type guard function changes in two ways. - .. code-block:: python +* Type narrowing is applied in the negative ("else") case. - def is_str(val: str | int) -> TypeGuard[str]: # Stricter mode possible - return isinstance(val, str) +.. code-block:: python - def func(val: str | int): - if is_str(val): - # Type checkers can assume val is a 'str' in this branch - else: - # Type checkers can assume val is an 'int' in this branch + def is_str(val: str | int) -> TypeGuard[str]: + return isinstance(val, str) -* When the output type of the ``TypeGuard`` is a union, the type checker can - apply additional type narrowing based on the type of the first input - argument, eliminating union elements that are impossible given the input - argument type: + def func(val: str | int): + if not is_str(val): + reveal_type(val) # int - .. code-block:: python +* Additional type narrowing is applied in the positive "if" case if applicable. + +.. code-block:: python - def is_cardinal_direction(val: str) -> TypeGuard[Literal["N", "S", "E", "W"]]: - return val in ("N", "S", "E", "W") + def is_cardinal_direction(val: str) -> TypeGuard[Literal["N", "S", "E", "W"]]: + return val in ("N", "S", "E", "W") - def func(direction: Literal["NW", "E"]): - if is_cardinal_direction(direction): - # Literal["E"] The type cannot be "N", "S" or "W" here because of - # argument type - else: - # Literal["NW"] + def func(direction: Literal["NW", "E"]): + if is_cardinal_direction(direction): + reveal_type(direction) # "Literal[E]" + else: + reveal_type(direction) # "Literal[NW]" -This logic can be summed up in a table: +The type-theoretic rules for type narrowing are specificed in the following +table. ============ ======================= =================== -\ Non strict TypeGuard Strict TypeGuard +\ Non-strict type guard Strict type guard ============ ======================= =================== -Applies when R not consistent with P R consistent with P +Applies when R not consistent with I R consistent with I NP is .. :math:`R` :math:`A \land R` NN is .. :math:`A` :math:`A \land \neg{R}` ============ ======================= =================== -Intersection of types and type negation are not defined and are left up to the -type checker to decide on how to implement. Future extensions to the type -system may define this behavior though. +In practice, the theoretic types for strict type guards cannot be expressed +precisely in the Python type system. Type checkers should fall back on +practical approximations of these types. As a rule of thumb, a type checker +should use the same type narrowing logic -- and get results that are consistent +with -- its handling of "isinstance". This guidance allows for changes and +improvements if the type system is extended in the future. Additional Examples @@ -145,20 +180,23 @@ stricter semantics can be applied. .. code-block:: python - def is_a(x: Any) -> TypeGuard[A]: # Stricter TypeGuard possible - return isinstance(x, A) + # Stricter type guard semantics are used in this case because + # "str" is consistent with "Any" + def is_str(x: Any) -> TypeGuard[str]: + return isinstance(x, str) - def test(x: A | B): - if is_a(x): - # x is of type A here + def test(x: float | str): + if is_str(x): + reveal_type(x) # str else: - # x is of type B here + reveal_type(x) # float Backwards Compatibility ======================= -The new ``False`` case for a ``TypeGuard`` breaks backwards compatibility. +This PEP proposes to change the existing behavior of ``TypeGuard``. This has no +effect at runtime, but it does change the types evaluated by a type checker. .. code-block:: python @@ -167,43 +205,25 @@ The new ``False`` case for a ``TypeGuard`` breaks backwards compatibility. def func(val: int | str): if is_int(val): - # Type checker assumes int here + reveal_type(val) # "int" else: - # Type checker assumes str here - -A type checker will assume in the ``False`` case that the value is ``str``. - -This is a change in behavior from :pep:`647`. + reveal_type(val) # Previously "int | str", now "str" -Although this is different behavior, existing code would likely have further -checks for a ``str`` in the ``False`` case. Since the previous behavior was to -assume ``str | int``, existing code would have had to further refine the type -to get a type checker to behave the expected way. -Something like so: +This behavioral change results in different types evaluated by a type checker. +It could therefore produce new (or mask existing) type errors. -.. code-block:: python - - def is_int(val: int | str) -> TypeGuard[int]: - return isinstance(val, int) +Type checkers often improve narrowing logic or fix existing bugs in such logic, +so users of static typing will be used to this type of behavioral change. - def is_str(val: int | str) -> TypeGuard[int]: - return isinstance(val, int) - - def func(val: int | str): - if is_int(val): - # Code does something with int here - else is_str(val): # This check would have been necessary before - # Code does something with str here - - -This means existing code should continue to work with the ``False`` narrowing. - -As a proof of this concept, we ran the `experimental Pyright changes`__ -against `mypy primer`__ to see if there were any differences in the output. -Changing ``TypeGuard`` to be more strict had a small effect, -mostly indicating that some ``# type: ignore`` comments were no longer -necessary. +We also hypothesize that it is unlikely that existing typed Python code relies +on the current behavior of ``TypeGuard``. To validate our hypothesis, we +implemented the proposed change in pyright and ran this modified version on +roughly 25 typed code bases using `mypy primer`__ to see if there were any +differences in the output. As predicted, the behavioral change had minimal +impact. The only noteworthy change was that some ``# type: ignore`` comments +were no longer necessary, indicating that these code bases were already working +around the existing limitations of ``TypeGuard``. __ https://github.com/microsoft/pyright/pull/5832 __ https://github.com/hauntsaninja/mypy_primer @@ -211,7 +231,8 @@ __ https://github.com/hauntsaninja/mypy_primer Breaking change --------------- -It is possible to create a broken ``TypeGuard`` now: +It is possible for a user-defined type guard function to rely on the old +behavior. Such type guard functions could break with the new behavior. .. code-block:: python @@ -220,30 +241,39 @@ It is possible to create a broken ``TypeGuard`` now: def func(val: int | str): if is_positive_int(val): - # Type checker assumes int here + reveal_type(val) # "int" else: - # Type checker now assumes str incorrectly here + # With the older behavior, the type of "val" is evaluated as + # "int | str"; with the new behavior, the type is narrowed to + # "str", which is perhaps not what was intended. + reveal_type(val) -This case is now incorrect. However, we're confident that this is not a -real world scenario. The mypy primer run didn't show any similar uses cases. -Dynamic ``TypeGuard``\s are not a common pattern. +We think it is unlikley that such user-defined type guards exist in real-world +code. The mypy primer results didn't uncover any such cases. How to Teach This ================= -We assert that users unfamiliar with ``TypeGuard`` will expect the behavior -outlined in this PEP, therefore making ``TypeGuard`` easier to teach and -explain. +Users unfamiliar with ``TypeGuard`` are likely to expect the behavior outlined +in this PEP, therefore making ``TypeGuard`` easier to teach and explain. Reference Implementation ======================== -A reference `implementation`__ of this idea exists in Pyright. +A reference `implementation`__ of this idea exists in pyright. __ https://github.com/microsoft/pyright/commit/9a5af798d726bd0612cebee7223676c39cf0b9b0 +To enable the modified behavior, the configuration flag +``enableExperimentalFeatures`` must be set to true. This can be done on a +per-file basis by adding a comment: + +.. code-block:: python + + # pyright: enableExperimentalFeatures=true + Rejected Ideas ============== @@ -251,26 +281,25 @@ Rejected Ideas StrictTypeGuard --------------- -Originally a new ``StrictTypeGuard`` construct was proposed. A -``StrictTypeGuard`` would be similar to a ``TypeGuard`` except it would -explicitly state that output type was consistent [#isconsistent]_ with the -input type. Type checkers would validate that the output type was consistent -[#isconsistent]_ with the input type. - -See this comment: `StrictTypeGuard proposal`__ +A new ``StrictTypeGuard`` construct was proposed. This alternative form would +be similar to a ``TypeGuard`` except it would apply stricter type guard +semantics. It would also enforce that the return type was consistent +[#isconsistent]_ with the input type. See this thread for details: +`StrictTypeGuard proposal`__ __ https://github.com/python/typing/discussions/1013#discussioncomment-1966238 -This was rejected because for most cases it's not necessary. Most people assume -when the ``TypeGuard`` returns ``False``, that the input type has been narrowed -to its other type. Why not just change the specification to match their -assumptions? +This idea was rejected because it is unnecessary in most cases and added +unnecessary complexity. It would require the introduction of a new special +form, and developers would need to be educated about the subtle difference +between the two forms. TypeGuard with a second output type ----------------------------------- -Another idea was also proposed where a ``TypeGuard`` could have a second output -type. Something like so: +Another idea was proposed where ``TypeGuard`` could support a second optional +type argument that indicates the type that should be used for narrowing in the +negative ("else") case. .. code-block:: python @@ -278,13 +307,13 @@ type. Something like so: return isinstance(val, int) -The second output type tells a type checker what type is returned in the ``False`` -case. It was originally proposed `here`__. +This idea was proposed `here`__. __ https://github.com/python/typing/issues/996 -This idea was rejected because of the negative feedback it received. -Specifically this `thread`__. +It was rejected because it was considered too complicated and addressed only +one of the two main limitations of ``TypeGuard``. Refer to this `thread`__ for +the full discussion. __ https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL @@ -300,3 +329,4 @@ Copyright This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. + From 159c51928e56cc3f92f14f568b2c47606a9bf7b9 Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 15 Sep 2023 16:10:50 -0700 Subject: [PATCH 65/67] Fix build and linter --- peps/pep-0724.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index 14135b17da3..fa3f55c35c4 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -4,14 +4,14 @@ Author: Rich Chiodo , Eric Traut , Erik De Bonte , Sponsor: Jelle Zijlstra -Discussions-To: +Discussions-To: https://mail.python.org/archives/list/typing-sig@python.org/thread/7KZ2VUDXZ5UKAUHRNXBJYBENAYMT6WXN/ Status: Draft Type: Standards Track Topic: Typing Content-Type: text/x-rst Created: 28-Jul-2023 Python-Version: 3.13 -Post-History: https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ +Post-History: 30-Dec-2021 https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ Abstract @@ -225,7 +225,6 @@ impact. The only noteworthy change was that some ``# type: ignore`` comments were no longer necessary, indicating that these code bases were already working around the existing limitations of ``TypeGuard``. -__ https://github.com/microsoft/pyright/pull/5832 __ https://github.com/hauntsaninja/mypy_primer Breaking change From 678ce5f35f5ee9433daf2d22fe094a25532fc4dd Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 15 Sep 2023 16:15:36 -0700 Subject: [PATCH 66/67] Try fixing post history --- peps/pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index fa3f55c35c4..6ce966db817 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -11,7 +11,7 @@ Topic: Typing Content-Type: text/x-rst Created: 28-Jul-2023 Python-Version: 3.13 -Post-History: 30-Dec-2021 https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/ +Post-History: `30-Dec-2021 https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/`__ Abstract From f817e070165512c7d62e1136f66187c535777b8c Mon Sep 17 00:00:00 2001 From: rchiodo Date: Fri, 15 Sep 2023 16:20:56 -0700 Subject: [PATCH 67/67] Fix post history again --- peps/pep-0724.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0724.rst b/peps/pep-0724.rst index 6ce966db817..00d3eb7fd73 100644 --- a/peps/pep-0724.rst +++ b/peps/pep-0724.rst @@ -11,7 +11,7 @@ Topic: Typing Content-Type: text/x-rst Created: 28-Jul-2023 Python-Version: 3.13 -Post-History: `30-Dec-2021 https://mail.python.org/archives/list/typing-sig@python.org/thread/EMUD2D424OI53DCWQ4H5L6SJD2IXBHUL/`__ +Post-History: `30-Dec-2021 `__ Abstract