IncompleteInputError is undocumented

[JelleZijlstra]

#113745 added a new built-in exception IncompleteInputError, but it is not mentioned anywhere in the docs (except in the auto-generated list of classes) or in the What's New. It should be documented. cc @pablogsal

Linked PRs

• gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from stable ABI #119680
• [3.13] gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680, GH-120955) #120944
• gh-119521: Use PyAPI_DATA, not extern, for _PyExc_IncompleteInpu…tError #120955
• gh-119521: Remove _IncompleteInputError from the docs #120993
• [3.13] gh-119521: Remove _IncompleteInputError from the docs (GH-120993) #121076

[pablogsal]

I don't want to document this as this is an implementation detail of the parser specially tailored for the codeop module. It is only raised when a very specific undocumented flag is passed to the parser to support better what the codeop module needs to do and I don't think we want users to depend on it.

[JelleZijlstra]

It's a builtin though. If we don't want this to be public, we shouldn't have put it in the builtins.

[JelleZijlstra]

Possibly the new exception should be given a dunder name (since dunder names are reserved for the language), or placed in some private extension module (maybe _ast?).

[pablogsal]

It's a builtin though. If we don't want this to be public, we shouldn't have put it in the builtins.

That's a side effect of being a subclass of SyntaxError that's available via the normal subclass in exceptions.c mechanism in a way that generates directly a type that we can use in the parser, unfortunately. Adding this to a module is much more complicated due to a bootstrapping problem with the parser (I need to double check if this is 100% the case so take this with a pinch of salt)

[pablogsal]

We can try to prepend a underscore to the name for sure.

I can try to investigate if there is an easy way to move it to some other hidden place when used via Python as well but IIRC it's a bit complex

[pablogsal]

Bad news: I dedicated some time to investigate and seems prepending the symbol with an underscore fails in macOS because prepending the symbol with an underscore triggers some linkage failure (doing any other alterations to the symbol name does not). Also, exceptions cannot be easily declared outside the Objects/exceptions.c module and even if they can there is a bootstrapping problem with the parser. We can proceed in two ways:

• Remove the exception, make the parser raise some weird exception and somehow parse the message (very ugly and error prone).
• Document the exception and say is an implementation detail.
• Document the exception.
• Somehow investigate if there is a way to override this problem with linkage

[pablogsal]

I may be possible that I am missing something obvious, but this diff:

ndex 68d7985dac8..0f71b1dcdb3 100644
--- a/Include/pyerrors.h
+++ b/Include/pyerrors.h
@@ -108,7 +108,7 @@ PyAPI_DATA(PyObject *) PyExc_NotImplementedError;
 PyAPI_DATA(PyObject *) PyExc_SyntaxError;
 PyAPI_DATA(PyObject *) PyExc_IndentationError;
 PyAPI_DATA(PyObject *) PyExc_TabError;
-PyAPI_DATA(PyObject *) PyExc_IncompleteInputError;
+PyAPI_DATA(PyObject *) _PyExc_IncompleteInputError;
 PyAPI_DATA(PyObject *) PyExc_ReferenceError;
 PyAPI_DATA(PyObject *) PyExc_SystemError;
 PyAPI_DATA(PyObject *) PyExc_SystemExit;
diff --git a/Lib/test/test_stable_abi_ctypes.py b/Lib/test/test_stable_abi_ctypes.py
index c06c285c501..446df809e9c 100644
--- a/Lib/test/test_stable_abi_ctypes.py
+++ b/Lib/test/test_stable_abi_ctypes.py
@@ -267,7 +267,7 @@ SYMBOL_NAMES = (
     "PyExc_IOError",
     "PyExc_ImportError",
     "PyExc_ImportWarning",
-    "PyExc_IncompleteInputError",
+    "_PyExc_IncompleteInputError",
     "PyExc_IndentationError",
     "PyExc_IndexError",
     "PyExc_InterruptedError",
diff --git a/Misc/stable_abi.toml b/Misc/stable_abi.toml
index 77473662aaa..597e8d24084 100644
--- a/Misc/stable_abi.toml
+++ b/Misc/stable_abi.toml
@@ -2480,7 +2480,7 @@
 [function._Py_SetRefcnt]
     added = '3.13'
     abi_only = true
-[data.PyExc_IncompleteInputError]
+[data._PyExc_IncompleteInputError]
     added = '3.13'
 [function.PyList_GetItemRef]
     added = '3.13'
diff --git a/PC/python3dll.c b/PC/python3dll.c
index 86c88843089..d72c87e1e9e 100755
--- a/PC/python3dll.c
+++ b/PC/python3dll.c
@@ -839,7 +839,7 @@ EXPORT_DATA(PyExc_FutureWarning)
 EXPORT_DATA(PyExc_GeneratorExit)
 EXPORT_DATA(PyExc_ImportError)
 EXPORT_DATA(PyExc_ImportWarning)
-EXPORT_DATA(PyExc_IncompleteInputError)
+EXPORT_DATA(_PyExc_IncompleteInputError)
 EXPORT_DATA(PyExc_IndentationError)
 EXPORT_DATA(PyExc_IndexError)
 EXPORT_DATA(PyExc_InterruptedError)
diff --git a/Parser/pegen.c b/Parser/pegen.c
index 3d3e6455940..ad462d25fda 100644
--- a/Parser/pegen.c
+++ b/Parser/pegen.c
@@ -844,7 +844,7 @@ _PyPegen_run_parser(Parser *p)
     if (res == NULL) {
         if ((p->flags & PyPARSE_ALLOW_INCOMPLETE_INPUT) &&  _is_end_of_source(p)) {
             PyErr_Clear();
-            return _PyPegen_raise_error(p, PyExc_IncompleteInputError, 0, "incomplete input");
+            return _PyPegen_raise_error(p, _PyExc_IncompleteInputError, 0, "incomplete input");
         }
         if (PyErr_Occurred() && !PyErr_ExceptionMatches(PyExc_SyntaxError)) {
             return NULL;
diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv
index 8b6fe94e3af..b62f89a56ab 100644
--- a/Tools/c-analyzer/cpython/globals-to-fix.tsv
+++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv
@@ -200,7 +200,7 @@ Objects/exceptions.c        -       _PyExc_AttributeError   -
 Objects/exceptions.c   -       _PyExc_SyntaxError      -
 Objects/exceptions.c   -       _PyExc_IndentationError -
 Objects/exceptions.c   -       _PyExc_TabError -
-Objects/exceptions.c   -       _PyExc_IncompleteInputError     -
+Objects/exceptions.c   -       __PyExc_IncompleteInputError    -
 Objects/exceptions.c   -       _PyExc_LookupError      -
 Objects/exceptions.c   -       _PyExc_IndexError       -
 Objects/exceptions.c   -       _PyExc_KeyError -
@@ -266,7 +266,7 @@ Objects/exceptions.c        -       PyExc_AttributeError    -
 Objects/exceptions.c   -       PyExc_SyntaxError       -
 Objects/exceptions.c   -       PyExc_IndentationError  -
 Objects/exceptions.c   -       PyExc_TabError  -
-Objects/exceptions.c   -       PyExc_IncompleteInputError      -
+Objects/exceptions.c   -       _PyExc_IncompleteInputError     -
 Objects/exceptions.c   -       PyExc_LookupError       -
 Objects/exceptions.c   -       PyExc_IndexError        -
 Objects/exceptions.c   -       PyExc_KeyError  -
diff --git a/Objects/exceptions.c b/Objects/exceptions.c
index f9cd577c1c1..01c72956a8c 100644
--- a/Objects/exceptions.c
+++ b/Objects/exceptions.c
@@ -2606,9 +2606,9 @@ MiddlingExtendsException(PyExc_IndentationError, TabError, SyntaxError,
                          "Improper mixture of spaces and tabs.");

 /*
- *    IncompleteInputError extends SyntaxError
+ *    _IncompleteInputError extends SyntaxError
  */
-MiddlingExtendsException(PyExc_SyntaxError, IncompleteInputError, SyntaxError,
+MiddlingExtendsException(PyExc_SyntaxError, _IncompleteInputError, SyntaxError,
                          "incomplete input.");

 /*
@@ -3675,7 +3675,7 @@ static struct static_exception static_exceptions[] = {

     // Level 4: Other subclasses
     ITEM(IndentationError), // base: SyntaxError(Exception)
-    ITEM(IncompleteInputError), // base: SyntaxError(Exception)
+    ITEM(_IncompleteInputError), // base: SyntaxError(Exception)
     ITEM(IndexError),  // base: LookupError(Exception)
     ITEM(KeyError),  // base: LookupError(Exception)
     ITEM(ModuleNotFoundError), // base: ImportError(Exception)

fails with:

Undefined symbols for architecture arm64:
  "__PyExc_IncompleteInputError", referenced from:
      __PyPegen_run_parser in pegen.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Programs/_freeze_module] Error 1

[JelleZijlstra]

We only want the Python-visible name to have an underscore (or two) though, the C name can stay. So I think you could manually expand the MiddlingExtendsException macro and use a different name for the C type name and the Python-visible __name__.

[pablogsal]

We only want the Python-visible name to have an underscore (or two) though, the C name can stay. So I think you could manually expand the MiddlingExtendsException macro and use a different name for the C type name and the Python-visible __name__.

I was a bit reluctant to add a new macro just for this but it seems that there is no much around this unfortunately

[pablogsal]

I found a better way to solve this problem!

[encukou]

The rename is merged, but looking at the issues again, I think we should also remove _IncompleteInputError from the docs.