From 02e9edfbe366a5ce7c53965848638b2893f80245 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Mon, 21 Jun 2021 13:15:54 -0400 Subject: [PATCH 01/46] __main__docs: intro and first secton --- Doc/library/__main__.rst | 99 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 95 insertions(+), 4 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index a64faf1bbe3c84..5bfdd84d75c0c0 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -1,12 +1,103 @@ - -:mod:`__main__` --- Top-level script environment -================================================ +:mod:`__main__` --- CLIs, import-time behavior, and ``__name__ == ‘__main__’`` +================================================================================= .. module:: __main__ - :synopsis: The environment where the top-level script is run. + :synopsis: CLIs, import-time behavior, and ``__name__ == ‘__main__’`` -------------- +The concept of ``__main__`` in Python can be confusing at first. It is not a +single mechanism in the language, but in fact is part of two quite different +constructs: + +1. The ``__main__.py`` file in a Python module; see :ref:`___main__.py`. +2. The very common ``__name__ == '__main__'`` construct. + +The term ``__main__`` in both of these use cases does indeed point towards a +common design philosophy, which hopefully will become clear. Nonetheless, each +will be documented separately in the following two sections. + + +.. ___main__.py: + +``__main__.py`` in Python Packages +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you are not familiar with Python packages, see section :ref:`tut-packages`. +Most commonly, the :file:`__main__.py` file in a Python package is used to +provide a command line interface (CLI) for that package. Consider the +following hypothetical package, "bandclass":: + + project_root + └── bandclass + ├── __init__.py + ├── __main__.py + ├── parent.py + ├── section.py + └── student.py + +``__main__.py`` will be excecuted when the package itself is invoked directly +from the command line using the ``-m`` flag. For example:: + + python3 -m bandclass + +In this example, ``__main__.py`` might be used to provide a CLI for the +teacher to get information. In combination with :mod:`argparse`, +``__main__.py`` becomes a beautiful way to add command line functionality +to your packages. + +For a very popular example of this usage pattern in our standard library, +see :mod:`http.server`, and its' invocation via +``python3 -m http.server [directory]`` + + + + + + + + + +.. REDRAFT PLAN + +There have been many complaints about the shortcoming of the documentation +towards informing users about __main__. Both the popular __name__ == '__main__' construct, and the role of __main__.py in a python module. + +bpo-17359 +bpo-24632 +bpo-38452 + +I propose a broad overhaul of Doc/library/__main__.rst to address these +shortcomings and to provide a single source of truth on __main__ (in +general!). This is an appropriate place to put this information. +Both the __name__ == '__main__' and fooModule/__main__.py +constructs reasonably fall under the category of “Python Runtime Services,” +because they both control the way that programs run depending on how they are +used (command-line versus import versus running directly). + +The new Doc/library/__main__.rst should have a new synopsis of, “CLIs, +import-time behavior, and if __name__ == ‘__main__’”, reflecting its new and +broader focus. + +Additionally, the new docs should have the following distinct sections: + + Differentiating between __name__ == ‘__main__’ and __main.__.py + __main__.py and the -m flag (this is roughly what is there already, although + it’s not as descriptive as it should be). + __name__ and the if __name__ == '__main__' construct. + +If there is interest, I would be happy to open uptake this work on as soon as there is +consensus around this plan. I’m looking forward to hearing what you think! + + + + + + + +.. OLD DOCUMENTATION + + ``'__main__'`` is the name of the scope in which top-level code executes. A module's __name__ is set equal to ``'__main__'`` when read from standard input, a script, or from an interactive prompt. From c95f69ba2303b37c9dbcfb56b2d4c0dc1353ce6e Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 22 Jun 2021 23:30:33 -0400 Subject: [PATCH 02/46] bpo-44494: rewrite of Doc/library/__main__.rst (first draft) * Related bpo's: 17359, 24632, 38452 --- Doc/library/__main__.rst | 145 ++++++++++++++++++++------------------- 1 file changed, 74 insertions(+), 71 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 5bfdd84d75c0c0..a1169d4e8799cf 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -4,113 +4,116 @@ .. module:: __main__ :synopsis: CLIs, import-time behavior, and ``__name__ == ‘__main__’`` +.. sectionauthor:: Jack DeVries + -------------- The concept of ``__main__`` in Python can be confusing at first. It is not a single mechanism in the language, but in fact is part of two quite different constructs: -1. The ``__main__.py`` file in a Python module; see :ref:`___main__.py`. -2. The very common ``__name__ == '__main__'`` construct. +1. The ``__main__.py`` file in a Python packages +2. The ``__name__ == '__main__'`` construct -The term ``__main__`` in both of these use cases does indeed point towards a -common design philosophy, which hopefully will become clear. Nonetheless, each -will be documented separately in the following two sections. +Each of these mechanisms are related to Python modules: both how users interact +with them as well as how they interact with each other. See section +:ref:`tut-modules`. -.. ___main__.py: +.. _main.py: ``__main__.py`` in Python Packages ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If you are not familiar with Python packages, see section :ref:`tut-packages`. -Most commonly, the :file:`__main__.py` file in a Python package is used to -provide a command line interface (CLI) for that package. Consider the -following hypothetical package, "bandclass":: - - project_root - └── bandclass - ├── __init__.py - ├── __main__.py - ├── parent.py - ├── section.py - └── student.py - -``__main__.py`` will be excecuted when the package itself is invoked directly -from the command line using the ``-m`` flag. For example:: - - python3 -m bandclass - -In this example, ``__main__.py`` might be used to provide a CLI for the -teacher to get information. In combination with :mod:`argparse`, -``__main__.py`` becomes a beautiful way to add command line functionality -to your packages. - -For a very popular example of this usage pattern in our standard library, -see :mod:`http.server`, and its' invocation via -``python3 -m http.server [directory]`` - - - +Most commonly, the ``__main__.py`` file is used to provide a command line +interface for a package. Consider the following hypothetical package, +"bandclass": +.. code-block:: text + bandclass + ├── __init__.py + ├── __main__.py + ├── parent.py + └── student.py +``__main__.py`` will be executed when the package itself is invoked +directly from the command line using the ``-m`` flag. For example:: + python3 -m bandclass +This command will cause ``__main__.py`` to run. How you utilize this +mechanism will depend on the nature of the package you are writing, but +in this hypothetical case, it might make sense to allow the teacher to search +for students or parents using :mod:`argparse`:: -.. REDRAFT PLAN + # bandclass/__main__.py -There have been many complaints about the shortcoming of the documentation -towards informing users about __main__. Both the popular __name__ == '__main__' construct, and the role of __main__.py in a python module. + import argparse + import sys -bpo-17359 -bpo-24632 -bpo-38452 + from .parent import Parents + from .student import Students -I propose a broad overhaul of Doc/library/__main__.rst to address these -shortcomings and to provide a single source of truth on __main__ (in -general!). This is an appropriate place to put this information. -Both the __name__ == '__main__' and fooModule/__main__.py -constructs reasonably fall under the category of “Python Runtime Services,” -because they both control the way that programs run depending on how they are -used (command-line versus import versus running directly). + parser = argparse.ArgumentParser() + parser.add_argument('--student', + help="lookup a student and print their information") + parser.add_argument('--parent', + help="lookup a parent and print their information") -The new Doc/library/__main__.rst should have a new synopsis of, “CLIs, -import-time behavior, and if __name__ == ‘__main__’”, reflecting its new and -broader focus. + args = parser.parse_args() -Additionally, the new docs should have the following distinct sections: + if args.student and student := Students.find(args.student): + print(student) + sys.exit('Student found') + elif args.parent and parent := Parents.find(args.parent): + print(parent) + sys.exit('Parent found') + else: + print('Result not found') + sys.exit(args.print_help()) - Differentiating between __name__ == ‘__main__’ and __main.__.py - __main__.py and the -m flag (this is roughly what is there already, although - it’s not as descriptive as it should be). - __name__ and the if __name__ == '__main__' construct. -If there is interest, I would be happy to open uptake this work on as soon as there is -consensus around this plan. I’m looking forward to hearing what you think! +For a very popular example of this usage pattern in our standard library, see +:mod:`http.server`, and its' invocation via ``python3 -m http.server +[directory]``. +``__name__ == '__main__'`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +``__name__`` is a special identifier which is always defined in Python. If a +Python module has been imported, ``__name__`` will be the same as the module +name (which is typically the file name without the ``.py`` stem). However, consider a +Python program being called directly from the command line like this:: + python3 bandclass/parent.py +In this case, ``__name__`` would be set to ``'__main__'`` inside of +``parent.py``. ``parent.py`` may have several functions defined inside of it, +but the ``__name__ == '__main__'`` statement can be used to test whether the +module has been invoked by the user directly. In response, Python modules can +behave differently when they are executed directly from the command line:: + # bandclass/parent.py -.. OLD DOCUMENTATION + def list_all(): + return InformationSystem.get_all('parents') + ... + if __name__ == '__main__': + print(list_all()) +The print function will run when this file is invoked directly, but not +if it is imported in ``student.py`` like so:: -``'__main__'`` is the name of the scope in which top-level code executes. -A module's __name__ is set equal to ``'__main__'`` when read from -standard input, a script, or from an interactive prompt. + # bandclass/student.py -A module can discover whether or not it is running in the main scope by -checking its own ``__name__``, which allows a common idiom for conditionally -executing code in a module when it is run as a script or with ``python --m`` but not when it is imported:: + from .parent import list_all - if __name__ == "__main__": - # execute only if run as a script - main() +In Python, imports cause the target module to be executed in its entirety; but +inside of ``parent.py``, ``__name__ == 'parent'`` when the Python module is +being executed in the context of an import. Therefore, the print statement in +the ``if __name__ == '__main__'`` block will not run. -For a package, the same effect can be achieved by including a -``__main__.py`` module, the contents of which will be executed when the -module is run with ``-m``. +For more information, see section :ref:`tut-modulesasscripts`. From 235e86638d3cf62f4aa2a7b4243bd2a778d526bd Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 22 Jun 2021 23:37:42 -0400 Subject: [PATCH 03/46] bpo-44494: add blurb --- .../next/Documentation/2021-06-22-23-37-38.bpo-44494._hpVPb.rst | 1 + 1 file changed, 1 insertion(+) create mode 100644 Misc/NEWS.d/next/Documentation/2021-06-22-23-37-38.bpo-44494._hpVPb.rst diff --git a/Misc/NEWS.d/next/Documentation/2021-06-22-23-37-38.bpo-44494._hpVPb.rst b/Misc/NEWS.d/next/Documentation/2021-06-22-23-37-38.bpo-44494._hpVPb.rst new file mode 100644 index 00000000000000..ce49b32d99168d --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2021-06-22-23-37-38.bpo-44494._hpVPb.rst @@ -0,0 +1 @@ +Rewrite and significant expansion of ``Doc/library/__main__.rst``. From a292ab6714ebb2c05818b165cea3c74add35c3b3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?G=C3=A9ry=20Ogam?= Date: Sun, 30 Jun 2019 18:52:56 +0200 Subject: [PATCH 04/46] Update __main__.rst --- Doc/library/__main__.rst | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index a64faf1bbe3c84..727626d0af0d29 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -1,25 +1,24 @@ - -:mod:`__main__` --- Top-level script environment -================================================ +:mod:`__main__` --- Top-level code environment +============================================== .. module:: __main__ - :synopsis: The environment where the top-level script is run. + :synopsis: The environment where top-level code is run. -------------- -``'__main__'`` is the name of the scope in which top-level code executes. -A module's __name__ is set equal to ``'__main__'`` when read from -standard input, a script, or from an interactive prompt. +``'__main__'`` is the name of the environment where top-level code is run. A +module's ``__name__`` is set equal to ``'__main__'`` when the module is run +from the file system, from standard input or from the module namespace (with +the :option:`-m` command line switch), but not when it is imported. -A module can discover whether or not it is running in the main scope by +A module can discover whether or not it is running in the main environment by checking its own ``__name__``, which allows a common idiom for conditionally -executing code in a module when it is run as a script or with ``python --m`` but not when it is imported:: +executing code in a module when it is not imported:: + # Execute only if the module is not imported. if __name__ == "__main__": - # execute only if run as a script main() -For a package, the same effect can be achieved by including a -``__main__.py`` module, the contents of which will be executed when the -module is run with ``-m``. +For a package, the same effect can be achieved by including a __main__.py +module, the contents of which will be executed when the package is run from the +file system or from the module namespace, but not when it is imported. From d7a199949a13b7125d352249b7c89f4be34794fa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?G=C3=A9ry=20Ogam?= Date: Sun, 30 Jun 2019 19:22:45 +0200 Subject: [PATCH 05/46] Update __main__.rst --- Doc/library/__main__.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 727626d0af0d29..e1632e9e5f30cb 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -9,7 +9,8 @@ ``'__main__'`` is the name of the environment where top-level code is run. A module's ``__name__`` is set equal to ``'__main__'`` when the module is run from the file system, from standard input or from the module namespace (with -the :option:`-m` command line switch), but not when it is imported. +the :option:`-m` command line switch or the :func:`runpy.run_module` function), +but not when it is imported. A module can discover whether or not it is running in the main environment by checking its own ``__name__``, which allows a common idiom for conditionally From 8398f0810ab4869bdcee1c11f72a44534449d91f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?G=C3=A9ry=20Ogam?= Date: Wed, 16 Sep 2020 22:46:29 +0200 Subject: [PATCH 06/46] =?UTF-8?q?Take=20Steven=20d=E2=80=99Aprano=E2=80=99?= =?UTF-8?q?s=20review=20into=20account?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- Doc/library/__main__.rst | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index e1632e9e5f30cb..118b89acf89d03 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -7,19 +7,20 @@ -------------- ``'__main__'`` is the name of the environment where top-level code is run. A -module's ``__name__`` is set equal to ``'__main__'`` when the module is run -from the file system, from standard input or from the module namespace (with -the :option:`-m` command line switch or the :func:`runpy.run_module` function), -but not when it is imported. +module's ``__name__`` is set equal to ``'__main__'`` when the module is +initialized from an interactive prompt, from standard input, from a file +argument, from a :option:`-c` argument or from a :option:`-m` argument, but +not when it is initialized from an import statement. A module can discover whether or not it is running in the main environment by checking its own ``__name__``, which allows a common idiom for conditionally -executing code in a module when it is not imported:: +executing code when the module is not initialized from an import statement:: - # Execute only if the module is not imported. - if __name__ == "__main__": - main() + if __name__ == '__main__': + # Execute when the module is not initialized from an import statement. + main() For a package, the same effect can be achieved by including a __main__.py -module, the contents of which will be executed when the package is run from the -file system or from the module namespace, but not when it is imported. +module, the contents of which will be executed when the package is initialized +from a file argument or from a :option:`-m` argument, but not when it is +initialized from an import statement. From 1fcb2afda1b3c34785f11398f6c60054399f1339 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?G=C3=A9ry=20Ogam?= Date: Wed, 16 Sep 2020 22:47:55 +0200 Subject: [PATCH 07/46] Rewrap lines --- Doc/library/__main__.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 118b89acf89d03..76e72a75db6d0c 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -6,21 +6,21 @@ -------------- -``'__main__'`` is the name of the environment where top-level code is run. A -module's ``__name__`` is set equal to ``'__main__'`` when the module is -initialized from an interactive prompt, from standard input, from a file -argument, from a :option:`-c` argument or from a :option:`-m` argument, but -not when it is initialized from an import statement. +``'__main__'`` is the name of the environment where top-level code is run. A +module's ``__name__`` is set equal to ``'__main__'`` when the module is +initialized from an interactive prompt, from standard input, from a file +argument, from a :option:`-c` argument or from a :option:`-m` argument, but not +when it is initialized from an import statement. -A module can discover whether or not it is running in the main environment by -checking its own ``__name__``, which allows a common idiom for conditionally +A module can discover whether or not it is running in the main environment by +checking its own ``__name__``, which allows a common idiom for conditionally executing code when the module is not initialized from an import statement:: if __name__ == '__main__': # Execute when the module is not initialized from an import statement. main() -For a package, the same effect can be achieved by including a __main__.py -module, the contents of which will be executed when the package is initialized -from a file argument or from a :option:`-m` argument, but not when it is +For a package, the same effect can be achieved by including a __main__.py +module, the contents of which will be executed when the package is initialized +from a file argument or from a :option:`-m` argument, but not when it is initialized from an import statement. From 2bde06362e0521ecd49c67f0b319ff1c6adb6dca Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?G=C3=A9ry=20Ogam?= Date: Thu, 17 Sep 2020 07:28:30 +0200 Subject: [PATCH 08/46] Remove trailing whitespaces --- Doc/library/__main__.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 76e72a75db6d0c..cb7233b6116b0e 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -6,21 +6,21 @@ -------------- -``'__main__'`` is the name of the environment where top-level code is run. A -module's ``__name__`` is set equal to ``'__main__'`` when the module is -initialized from an interactive prompt, from standard input, from a file -argument, from a :option:`-c` argument or from a :option:`-m` argument, but not +``'__main__'`` is the name of the environment where top-level code is run. A +module's ``__name__`` is set equal to ``'__main__'`` when the module is +initialized from an interactive prompt, from standard input, from a file +argument, from a :option:`-c` argument or from a :option:`-m` argument, but not when it is initialized from an import statement. -A module can discover whether or not it is running in the main environment by -checking its own ``__name__``, which allows a common idiom for conditionally +A module can discover whether or not it is running in the main environment by +checking its own ``__name__``, which allows a common idiom for conditionally executing code when the module is not initialized from an import statement:: if __name__ == '__main__': # Execute when the module is not initialized from an import statement. main() -For a package, the same effect can be achieved by including a __main__.py -module, the contents of which will be executed when the package is initialized -from a file argument or from a :option:`-m` argument, but not when it is +For a package, the same effect can be achieved by including a __main__.py +module, the contents of which will be executed when the package is initialized +from a file argument or from a :option:`-m` argument, but not when it is initialized from an import statement. From 4c60f2c17fe780eac498e5b4f334de83104b3914 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 29 Jun 2021 16:34:01 -0400 Subject: [PATCH 09/46] mention runpy --- Doc/library/__main__.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 1d2ec283cf1e5c..e8912be9456959 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -61,10 +61,11 @@ directly from the command line using the :option:`-m` flag. For example:: python3 -m bandclass -This command will cause ``__main__.py`` to run. How you utilize this -mechanism will depend on the nature of the package you are writing, but -in this hypothetical case, it might make sense to allow the teacher to search -for students or parents using :mod:`argparse`:: +This command will cause ``__main__.py`` to run. For more details about the +:option:`-m` flag, see :mod:`runpy`. How you utilize this mechanism will depend +on the nature of the package you are writing, but in this hypothetical case, it +might make sense to allow the teacher to search for students or parents using +:mod:`argparse`:: # bandclass/__main__.py From 2b5f7104d75a2f4dd91256072ba30324bd403a91 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 29 Jun 2021 17:26:45 -0400 Subject: [PATCH 10/46] add "design patterns" section, fix section title hierarchies --- Doc/library/__main__.rst | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index e8912be9456959..4cc489b428ae38 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -18,7 +18,7 @@ users interact with them as well as how they interact with each other. See ``__name__ == '__main__'`` -^^^^^^^^^^^^^^^^^^^^^^^^^^ +--------------------------- ``'__main__'`` is the name of the environment where top-level code is run. "Top-level code" means when a Python module is initialized from an interactive @@ -39,9 +39,39 @@ an import statement:: # Execute when the module is not initialized from an import statement. ... +Design Patterns +^^^^^^^^^^^^^^^ + +Putting as few statements as possible in the block below ``if __name___ == +'__main__'`` can improve the clarity of your code. Most often, a function named +``main`` encapuslates the program's "main" behavior, creating this pattern:: + + # echo.py + + import sys + + def main(phrase: str): + "Print the string to standard output" + print(phrase) + + if __name__ == '__main__': + main(' '.join(sys.argv)) + +This has the added benefit of the ``main`` function itself being importable +elsewhere:: + + # elsewhere.py + + import sys + + from echo import main as echo_main + + def echo_platform(): + echo_main(sys.platform) + ``__main__.py`` in Python Packages -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +---------------------------------- If you are not familiar with Python packages, see section :ref:`tut-packages`. Most commonly, the ``__main__.py`` file is used to provide a command line From 7e495d7bba0d4cd0685fc66bc179a3bd88359532 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 29 Jun 2021 17:49:57 -0400 Subject: [PATCH 11/46] add sentence about console_scripts --- Doc/library/__main__.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 4cc489b428ae38..8b226836315b20 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -126,7 +126,10 @@ might make sense to allow the teacher to search for students or parents using Note that there is no reason to use the ``if __name__ == '__main__'`` statement in ``__main__.py`` itself. There is no reason for any other file to import something from ``__main__.py``, and therefore, ``__name__`` will always be -``'__main__'``; in most cases it would be a redundant statement. +``'__main__'``; in most cases it would be a redundant statement. There are +exceptions to this norm, though. For example, if you have explicitly identified +``__main__`` as a console script entry point in :file:`setup.py`. See section +:ref:`entry-points`. For a very popular example of a package using ``__main__.py`` in our standard library, see :mod:`venv`, and its' invocation via ``python3 -m From 56afeaa11bab9d70090e6219d74d9bbc0d4d31fa Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 29 Jun 2021 21:19:59 -0400 Subject: [PATCH 12/46] misc formatting; change "Design Patterns" to "Idiomatic Usage" --- Doc/library/__main__.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 8b226836315b20..5cebaaf5d4f5de 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -10,7 +10,7 @@ In Python, ``__main__`` is not a single mechanism in the language, but in fact is part of two quite different constructs: 1. The ``__name__ == '__main__'`` statement -2. The ``__main__.py`` file in a Python packages +2. The ``__main__.py`` file in Python packages Each of these mechanisms are related to Python :ref:`tut-modules`; both how users interact with them as well as how they interact with each other. See @@ -39,25 +39,25 @@ an import statement:: # Execute when the module is not initialized from an import statement. ... -Design Patterns +Idiomatic Usage ^^^^^^^^^^^^^^^ Putting as few statements as possible in the block below ``if __name___ == '__main__'`` can improve the clarity of your code. Most often, a function named -``main`` encapuslates the program's "main" behavior, creating this pattern:: +*main* encapsulates the program's primary behavior, creating this pattern:: # echo.py import sys def main(phrase: str): - "Print the string to standard output" - print(phrase) + "Print the string to standard output" + print(phrase) if __name__ == '__main__': main(' '.join(sys.argv)) -This has the added benefit of the ``main`` function itself being importable +This has the added benefit of the *main* function itself being importable elsewhere:: # elsewhere.py From 2a398ef21eedff5c88c39df9df599177ba35d952 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 29 Jun 2021 21:21:16 -0400 Subject: [PATCH 13/46] add section about sys.exit(main()) convention --- Doc/library/__main__.rst | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 5cebaaf5d4f5de..d0d8fc52dfdc2f 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -67,7 +67,32 @@ elsewhere:: from echo import main as echo_main def echo_platform(): - echo_main(sys.platform) + echo_main(sys.platform) + +The spirit of this design is inherited from the C programming language, where +the function whose name is *main* is the entry-point of a program. In C, +*main* also returns an integer, which becomes the exit code of the process. +Zero typically indicates successful termination, and other codes indicate some +type of failure. :func:`sys.exit` provides the API for exiting with an +explicit exit code. A popular convention in Python is for *main* functions to +also return an integer which is then passed directly into :func:`sys.exit`, +making it the exit code of the process:: + + # first_char.py + + import sys + + def main(argv: list[str]) -> int: + try: + print(f'The first character is: {argv[1][0]}') + return 0 + except IndexError: + print('ERROR: first character could not be found. ' + 'Did you pass an argument?') + return 1 + + if __name__ == '__main__': + sys.exit(main(sys.argv)) ``__main__.py`` in Python Packages From 1a9956cb9d2dbb2672ebcd4003375b1ae2cb4441 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Wed, 30 Jun 2021 12:13:14 -0400 Subject: [PATCH 14/46] make last paragraph 'idiomatic usage'; add comment about maybe deleting --- Doc/library/__main__.rst | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index d0d8fc52dfdc2f..20797b1d8b9705 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -148,13 +148,37 @@ might make sense to allow the teacher to search for students or parents using print('Result not found') sys.exit(args.print_help()) -Note that there is no reason to use the ``if __name__ == '__main__'`` statement -in ``__main__.py`` itself. There is no reason for any other file to import -something from ``__main__.py``, and therefore, ``__name__`` will always be -``'__main__'``; in most cases it would be a redundant statement. There are -exceptions to this norm, though. For example, if you have explicitly identified -``__main__`` as a console script entry point in :file:`setup.py`. See section -:ref:`entry-points`. + + +Idiomatic Usage +^^^^^^^^^^^^^^^ + +.. should the first paragraph of this section be removed entirely? I see that + this suggestion conflicts with setuptools's docs, where they do use + if __name__ == '__main__' in __main__.py files + + (https://setuptools.readthedocs.io/en/latest/userguide/entry_point.html) + + However, I still think that the suggestion makes sense at face value. This + is my reasoning: + + > It seems to me that it is almost always redundant, except in the case of + > console scripts where `__name__` would be `package.__main__`. Even then, + > wouldn't you **not** want your code to be under a `__name__ == '__main__'` + > block in that case? If it were, the code you'd want to run wouldn't run when + > invoked as a console script. To me, this seems like another reason to tell + > users _not_ to guard code in `__main__.py` under an `if __name__ == + > '__main__'` block. `__main__.py` should always run from top-to-bottom; is + > that not the case? + + +Note that it may not be necessary to use the ``if __name__ == '__main__'`` +statement in ``__main__.py`` itself. There is no reason for any other file to +import something from ``__main__.py``. ``__main__.py`` will normally always be +executed as the main program; therefore, ``__name__`` will always be +``'__main__'``. There are exceptions to this norm, though. For example, if you +have explicitly identified ``__main__`` as a console script entry point in +:file:`setup.py`. See section :ref:`entry-points`. For a very popular example of a package using ``__main__.py`` in our standard library, see :mod:`venv`, and its' invocation via ``python3 -m From c4b5ceae8e6a548b3f5fdbad54ef260ffb8e07c7 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Wed, 30 Jun 2021 12:33:55 -0400 Subject: [PATCH 15/46] fix linting error (default context used in comment) --- Doc/library/__main__.rst | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 20797b1d8b9705..7bcbb6aac67e2e 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -153,7 +153,8 @@ might make sense to allow the teacher to search for students or parents using Idiomatic Usage ^^^^^^^^^^^^^^^ -.. should the first paragraph of this section be removed entirely? I see that +.. + should the first paragraph of this section be removed entirely? I see that this suggestion conflicts with setuptools's docs, where they do use if __name__ == '__main__' in __main__.py files @@ -162,14 +163,14 @@ Idiomatic Usage However, I still think that the suggestion makes sense at face value. This is my reasoning: - > It seems to me that it is almost always redundant, except in the case of - > console scripts where `__name__` would be `package.__main__`. Even then, - > wouldn't you **not** want your code to be under a `__name__ == '__main__'` - > block in that case? If it were, the code you'd want to run wouldn't run when - > invoked as a console script. To me, this seems like another reason to tell - > users _not_ to guard code in `__main__.py` under an `if __name__ == - > '__main__'` block. `__main__.py` should always run from top-to-bottom; is - > that not the case? + It seems to me that it is almost always redundant, except in the case of + console scripts where __name__ would be package.__main__. Even then, + wouldn't you **not** want your code to be under a __name__ == + '__main__' block in that case? If it were, the code you'd want to run + wouldn't run when invoked as a console script. To me, this seems like + another reason to tell users _not_ to guard code in __main__.py under + an if __name__ == '__main__' block. __main__.py should always run + from top-to-bottom; is that not the case? Note that it may not be necessary to use the ``if __name__ == '__main__'`` From 1f012b42653b3eac216a9e2f64728e2c6a17e82a Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Wed, 30 Jun 2021 15:43:21 -0400 Subject: [PATCH 16/46] revise example so that main() does not take arguments --- Doc/library/__main__.rst | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 7bcbb6aac67e2e..e9fe117fb8576e 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -48,26 +48,32 @@ Putting as few statements as possible in the block below ``if __name___ == # echo.py + import shlex import sys - def main(phrase: str): - "Print the string to standard output" - print(phrase) + def echo(phrase: str): + # you can imagine that this dummy wrapper around print might be + # different and truly worth re-using in a real program. + print(phrase) + + def main(): + "Echo the string to standard output" + echo(shlex.join(sys.argv)) if __name__ == '__main__': - main(' '.join(sys.argv)) + main() -This has the added benefit of the *main* function itself being importable -elsewhere:: +This has the added benefit of the *echo* function itself being isolated and +importable elsewhere:: # elsewhere.py import sys - from echo import main as echo_main + from echo import echo def echo_platform(): - echo_main(sys.platform) + echo(sys.platform) The spirit of this design is inherited from the C programming language, where the function whose name is *main* is the entry-point of a program. In C, From f095362efb2df6ddbe5cd6a8af62f2d5e10393ca Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Wed, 30 Jun 2021 15:44:30 -0400 Subject: [PATCH 17/46] add console_scripts section, remove bad old example --- Doc/library/__main__.rst | 57 ++++++++++++++++++++++++++-------------- 1 file changed, 38 insertions(+), 19 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index e9fe117fb8576e..86955dfadcec94 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -75,30 +75,49 @@ importable elsewhere:: def echo_platform(): echo(sys.platform) -The spirit of this design is inherited from the C programming language, where -the function whose name is *main* is the entry-point of a program. In C, -*main* also returns an integer, which becomes the exit code of the process. -Zero typically indicates successful termination, and other codes indicate some -type of failure. :func:`sys.exit` provides the API for exiting with an -explicit exit code. A popular convention in Python is for *main* functions to -also return an integer which is then passed directly into :func:`sys.exit`, -making it the exit code of the process:: - # first_char.py +Packaging Considerations (``console_scripts``) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For detailed documentation on Python packaging, see +`setuptools. `__ + +*main* functions are often used to create command line tools by specifying them +as entry points for console scripts. When this is done, pip inserts the +function call into a template that looks like this:: + + #!/path/to/python3 + # -*- coding: utf-8 -*- + import re + import sys + from package.__main__ import main + if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit(main()) + +Notice the **last line.** The call to *main* is wrapped in :func:`sys.exit`. +When *main* is the entry point of a console_script, the expectation is that +your function will return some value acceptable as an input to +:func:`sys.exit`; typically, an integer or ``None`` (which is implicitly returned +if your function does not have a return statement). + +By proactively folloiwng this convention ourselves, our module will have the +same behavior when run directly (``python3 echo.py``) as it will have if we +later pacakge it as an console script entrypoint in a pip-installable package. +We can revise the :file:`echo.py` example from earlier to follow this +convention:: - import sys + # echo.py + ... - def main(argv: list[str]) -> int: - try: - print(f'The first character is: {argv[1][0]}') - return 0 - except IndexError: - print('ERROR: first character could not be found. ' - 'Did you pass an argument?') - return 1 + def main() -> int: # now, main returns an integer + "Echo the string to standard output" + echo(shlex.join(sys.argv)) + return 0 if __name__ == '__main__': - sys.exit(main(sys.argv)) + # now, the integer returned from main is passed through to sys.exit + sys.exit(main()) ``__main__.py`` in Python Packages From c42b706387fc553876062b7f58c11bdeb7bee26e Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Wed, 30 Jun 2021 16:02:36 -0400 Subject: [PATCH 18/46] minor proofreading changes --- Doc/library/__main__.rst | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 86955dfadcec94..713005e0b56a57 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -12,9 +12,9 @@ is part of two quite different constructs: 1. The ``__name__ == '__main__'`` statement 2. The ``__main__.py`` file in Python packages -Each of these mechanisms are related to Python :ref:`tut-modules`; both how +Each of these mechanisms are related to Python modules; both how users interact with them as well as how they interact with each other. See -:ref:`tut-modules` for details. +section :ref:`tut-modules`. ``__name__ == '__main__'`` @@ -22,13 +22,14 @@ users interact with them as well as how they interact with each other. See ``'__main__'`` is the name of the environment where top-level code is run. "Top-level code" means when a Python module is initialized from an interactive -prompt, from standard input, from a file argument, from a :option:`-c` argument -or from a :option:`-m` argument, but **not** when it is initialized from an -import statement. In any of these situations, the module's ``__name__`` is set -equal to ``'__main__'``. The only other context in which Python code is run is -when it is imported through an import statement. In that case, ``__name__`` is -set equal to the module's name; usually the name of the file without the -``.py`` extension. +prompt, from standard input, from a file argument, from a :option:`-c` +argument, or from a :option:`-m` argument, but **not** when it is initialized +from an import statement. In any of these situations, the module's +``__name__`` is set equal to ``'__main__'``. + +The only other context in which Python code is run is when it is imported +through an import statement. In that case, ``__name__`` is set equal to the +module's name: usually the name of the file without the ``.py`` extension. As a result, a module can discover whether or not it is running in the top-level environment by checking its own ``__name__``, which allows a common @@ -43,8 +44,8 @@ Idiomatic Usage ^^^^^^^^^^^^^^^ Putting as few statements as possible in the block below ``if __name___ == -'__main__'`` can improve the clarity of your code. Most often, a function named -*main* encapsulates the program's primary behavior, creating this pattern:: +'__main__'`` can improve code clarity. Most often, a function named *main* +encapsulates the program's primary behavior, creating this pattern:: # echo.py @@ -57,7 +58,7 @@ Putting as few statements as possible in the block below ``if __name___ == print(phrase) def main(): - "Echo the string to standard output" + "Echo the sys.argv to standard output" echo(shlex.join(sys.argv)) if __name__ == '__main__': From 7fe7f1c170077ac63d94f93c67333454c2681216 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Wed, 30 Jun 2021 16:07:20 -0400 Subject: [PATCH 19/46] implement changes suggest by @merwok --- Doc/library/__main__.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 713005e0b56a57..5656b6873b4953 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -2,7 +2,7 @@ ============================================================================== .. module:: __main__ - :synopsis: CLIs, import-time behavior, and ``__name__ == '__main__'`` + :synopsis: Command line interfaces, import-time behavior, and ``__name__ == '__main__'`` -------------- @@ -208,5 +208,5 @@ have explicitly identified ``__main__`` as a console script entry point in :file:`setup.py`. See section :ref:`entry-points`. For a very popular example of a package using ``__main__.py`` in our standard -library, see :mod:`venv`, and its' invocation via ``python3 -m +library, see :mod:`venv`, and its invocation via ``python3 -m venv [directory]``. From c063da1780cd975fbf1e1b5259d034fb1a8b0f1d Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Wed, 30 Jun 2021 17:10:23 -0400 Subject: [PATCH 20/46] fix: typos --- Doc/library/__main__.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 5656b6873b4953..35f988d197038a 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -102,9 +102,9 @@ your function will return some value acceptable as an input to :func:`sys.exit`; typically, an integer or ``None`` (which is implicitly returned if your function does not have a return statement). -By proactively folloiwng this convention ourselves, our module will have the +By proactively following this convention ourselves, our module will have the same behavior when run directly (``python3 echo.py``) as it will have if we -later pacakge it as an console script entrypoint in a pip-installable package. +later package it as an console script entry-point in a pip-installable package. We can revise the :file:`echo.py` example from earlier to follow this convention:: From 7c6b451c35787dd62f2431adebf548f9beb6b9d1 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Wed, 7 Jul 2021 14:23:28 -0400 Subject: [PATCH 21/46] fix wording, slim down example, add reference to relative import docs --- Doc/library/__main__.rst | 40 ++++++++++++---------------------------- Doc/tutorial/modules.rst | 2 ++ 2 files changed, 14 insertions(+), 28 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 35f988d197038a..112854b969419b 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -12,8 +12,8 @@ is part of two quite different constructs: 1. The ``__name__ == '__main__'`` statement 2. The ``__main__.py`` file in Python packages -Each of these mechanisms are related to Python modules; both how -users interact with them as well as how they interact with each other. See +Each of these mechanisms are related to Python modules; how +users interact with them and how they interact with each other. See section :ref:`tut-modules`. @@ -134,7 +134,6 @@ interface for a package. Consider the following hypothetical package, bandclass ├── __init__.py ├── __main__.py - ├── parent.py └── student.py ``__main__.py`` will be executed when the package itself is invoked @@ -145,35 +144,21 @@ directly from the command line using the :option:`-m` flag. For example:: This command will cause ``__main__.py`` to run. For more details about the :option:`-m` flag, see :mod:`runpy`. How you utilize this mechanism will depend on the nature of the package you are writing, but in this hypothetical case, it -might make sense to allow the teacher to search for students or parents using +might make sense to allow the teacher to search for students using :mod:`argparse`:: # bandclass/__main__.py - import argparse import sys + from .student import search_students - from .parent import Parents - from .student import Students - - parser = argparse.ArgumentParser() - parser.add_argument('--student', - help="lookup a student and print their information") - parser.add_argument('--parent', - help="lookup a parent and print their information") - - args = parser.parse_args() - - if args.student and student := Students.find(args.student): - print(student) - sys.exit('Student found') - elif args.parent and parent := Parents.find(args.parent): - print(parent) - sys.exit('Parent found') - else: - print('Result not found') - sys.exit(args.print_help()) + student_name = sys.argv[2] if len(sys.argv) >= 2 else '' + print('Found student: {search_students(student_name)}') +Note that ``from .student import search_students`` is an example of a relative +import. This import style must be used when referencing modules within a +package. For more details, see :ref:`tut-modules`; or, more specifically, +:ref:`intra-package-references`. Idiomatic Usage @@ -207,6 +192,5 @@ executed as the main program; therefore, ``__name__`` will always be have explicitly identified ``__main__`` as a console script entry point in :file:`setup.py`. See section :ref:`entry-points`. -For a very popular example of a package using ``__main__.py`` in our standard -library, see :mod:`venv`, and its invocation via ``python3 -m -venv [directory]``. +For an example of a package using ``__main__.py`` in our standard library, see +:mod:`venv`, and its invocation via ``python3 -m venv [directory]``. diff --git a/Doc/tutorial/modules.rst b/Doc/tutorial/modules.rst index af595e5ca04d7e..a495c50cbde880 100644 --- a/Doc/tutorial/modules.rst +++ b/Doc/tutorial/modules.rst @@ -533,6 +533,8 @@ importing module needs to use submodules with the same name from different packages. +.. _intra-package-references: + Intra-package References ------------------------ From 06bcb09e0b5f30776463675e8bf892f8c248e720 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Wed, 21 Jul 2021 13:03:41 -0400 Subject: [PATCH 22/46] respond to review from @pradyunsg --- Doc/library/__main__.rst | 36 +++++++++++++----------------------- 1 file changed, 13 insertions(+), 23 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 112854b969419b..306220214e6464 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -80,27 +80,20 @@ importable elsewhere:: Packaging Considerations (``console_scripts``) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -For detailed documentation on Python packaging, see -`setuptools. `__ +For detailed documentation on Python packaging, see the +`Python Packaging User Guide. `__ *main* functions are often used to create command line tools by specifying them as entry points for console scripts. When this is done, pip inserts the -function call into a template that looks like this:: - - #!/path/to/python3 - # -*- coding: utf-8 -*- - import re - import sys - from package.__main__ import main - if __name__ == '__main__': - sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) - sys.exit(main()) - -Notice the **last line.** The call to *main* is wrapped in :func:`sys.exit`. -When *main* is the entry point of a console_script, the expectation is that -your function will return some value acceptable as an input to -:func:`sys.exit`; typically, an integer or ``None`` (which is implicitly returned -if your function does not have a return statement). +function call into a template script, where the return value of *main* is +passed into sys.exit. For example:: + + sys.exit(main()) + +Since the call to *main* is wrapped in :func:`sys.exit`, the expectation is +that your function will return some value acceptable as an input to +:func:`sys.exit`; typically, an integer or ``None`` (which is implicitly +returned if your function does not have a return statement). By proactively following this convention ourselves, our module will have the same behavior when run directly (``python3 echo.py``) as it will have if we @@ -108,12 +101,9 @@ later package it as an console script entry-point in a pip-installable package. We can revise the :file:`echo.py` example from earlier to follow this convention:: - # echo.py - ... - def main() -> int: # now, main returns an integer - "Echo the string to standard output" - echo(shlex.join(sys.argv)) + "Echo the input to standard output" + print(shlex.join(sys.argv)) return 0 if __name__ == '__main__': From 647c47124719205e3762581ef043c8bef4acc38a Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Sat, 31 Jul 2021 22:02:13 -0400 Subject: [PATCH 23/46] add `import __main__` section --- Doc/library/__main__.rst | 44 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 41 insertions(+), 3 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 306220214e6464..bb881bd1653bb4 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -17,6 +17,8 @@ users interact with them and how they interact with each other. See section :ref:`tut-modules`. +.. _name_is_main: + ``__name__ == '__main__'`` --------------------------- @@ -106,9 +108,45 @@ convention:: print(shlex.join(sys.argv)) return 0 - if __name__ == '__main__': - # now, the integer returned from main is passed through to sys.exit - sys.exit(main()) + +``import __main__`` +------------------- + +All the values in the ``__main__`` namespace can be imported elsewhere in +Python packages. See section :ref:`name_is_main` for a list of where the +``__main__`` package is in different Python execution scenarios. + +Here is an example package that consumes the ``__main__`` namespace:: + + # namely.py + + import __main__ + + def did_user_define_their_name(): + return 'my_name' in dir(__main__) + + def print_user_name(): + if did_user_define_their_name(): + print(__main__.my_name) + else: + print('Tell us your name by defining the variable `my_name`!') + +The Python REPL is one example of a "top-level environment", so anything +defined in the REPL becomes part of the ``__main__`` package:: + + >>> import namely + >>> namely.did_user_define_their_name() + False + >>> namely.print_user_name() + Tell us your name by defining the variable `my_name`! + >>> my_name = 'David' + >>> namely.did_user_define_their_name() + True + >>> namely.print_user_name() + David + +The ``__main__`` package is used in the implementation of :mod:`pdb` and +:mod:`rlcompleter`. ``__main__.py`` in Python Packages From 457bbc918300ba9c0f45df8431ffae6b48769bc1 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Sat, 31 Jul 2021 22:04:16 -0400 Subject: [PATCH 24/46] revisions and proofreading --- Doc/library/__main__.rst | 47 ++++++++++++++++------------------------ 1 file changed, 19 insertions(+), 28 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index bb881bd1653bb4..44d2c1eaeb6db5 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -12,9 +12,8 @@ is part of two quite different constructs: 1. The ``__name__ == '__main__'`` statement 2. The ``__main__.py`` file in Python packages -Each of these mechanisms are related to Python modules; how -users interact with them and how they interact with each other. See -section :ref:`tut-modules`. +Each of these mechanisms are related to Python modules; how users interact with +them and how they interact with each other. See section :ref:`tut-modules`. .. _name_is_main: @@ -42,48 +41,45 @@ an import statement:: # Execute when the module is not initialized from an import statement. ... + Idiomatic Usage ^^^^^^^^^^^^^^^ Putting as few statements as possible in the block below ``if __name___ == '__main__'`` can improve code clarity. Most often, a function named *main* -encapsulates the program's primary behavior, creating this pattern:: +encapsulates the program's primary behavior:: # echo.py import shlex import sys + def echo(phrase: str): - # you can imagine that this dummy wrapper around print might be - # different and truly worth re-using in a real program. + """A dummy wrapper around print.""" + # for demonstration purposes, you can imagine that there is some + # valuable and reusable logic inside this function print(phrase) + def main(): - "Echo the sys.argv to standard output" + """Echo the input arguments to standard output""" echo(shlex.join(sys.argv)) + if __name__ == '__main__': - main() + sys.exit(main()) # next section explains the use of sys.exit This has the added benefit of the *echo* function itself being isolated and -importable elsewhere:: - - # elsewhere.py - - import sys - - from echo import echo - - def echo_platform(): - echo(sys.platform) +importable elsewhere. None of the code in ``echo.py`` will execute at +import-time. -Packaging Considerations (``console_scripts``) +Packaging Considerations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For detailed documentation on Python packaging, see the -`Python Packaging User Guide. `__ +`Python Packaging User Guide. `_ *main* functions are often used to create command line tools by specifying them as entry points for console scripts. When this is done, pip inserts the @@ -99,14 +95,9 @@ returned if your function does not have a return statement). By proactively following this convention ourselves, our module will have the same behavior when run directly (``python3 echo.py``) as it will have if we -later package it as an console script entry-point in a pip-installable package. -We can revise the :file:`echo.py` example from earlier to follow this -convention:: - - def main() -> int: # now, main returns an integer - "Echo the input to standard output" - print(shlex.join(sys.argv)) - return 0 +later package it as a console script entry-point in a pip-installable package. +That is why the ``echo.py`` example from earlier used the ``sys.exit(main())`` +convention. ``import __main__`` From 3d9b3b9315bca1f9471c52e8f8310ea26fda1ece Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Sat, 31 Jul 2021 22:05:14 -0400 Subject: [PATCH 25/46] eliminate opinionated section about idiomatic usage of `__main__.py` --- Doc/library/__main__.rst | 32 -------------------------------- 1 file changed, 32 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 44d2c1eaeb6db5..2fcae03b39e513 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -179,37 +179,5 @@ import. This import style must be used when referencing modules within a package. For more details, see :ref:`tut-modules`; or, more specifically, :ref:`intra-package-references`. - -Idiomatic Usage -^^^^^^^^^^^^^^^ - -.. - should the first paragraph of this section be removed entirely? I see that - this suggestion conflicts with setuptools's docs, where they do use - if __name__ == '__main__' in __main__.py files - - (https://setuptools.readthedocs.io/en/latest/userguide/entry_point.html) - - However, I still think that the suggestion makes sense at face value. This - is my reasoning: - - It seems to me that it is almost always redundant, except in the case of - console scripts where __name__ would be package.__main__. Even then, - wouldn't you **not** want your code to be under a __name__ == - '__main__' block in that case? If it were, the code you'd want to run - wouldn't run when invoked as a console script. To me, this seems like - another reason to tell users _not_ to guard code in __main__.py under - an if __name__ == '__main__' block. __main__.py should always run - from top-to-bottom; is that not the case? - - -Note that it may not be necessary to use the ``if __name__ == '__main__'`` -statement in ``__main__.py`` itself. There is no reason for any other file to -import something from ``__main__.py``. ``__main__.py`` will normally always be -executed as the main program; therefore, ``__name__`` will always be -``'__main__'``. There are exceptions to this norm, though. For example, if you -have explicitly identified ``__main__`` as a console script entry point in -:file:`setup.py`. See section :ref:`entry-points`. - For an example of a package using ``__main__.py`` in our standard library, see :mod:`venv`, and its invocation via ``python3 -m venv [directory]``. From dd68513f77c2c3cae706c56c975f552250ce048f Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Sat, 31 Jul 2021 22:21:38 -0400 Subject: [PATCH 26/46] proofread `__main__.py` section --- Doc/library/__main__.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 2fcae03b39e513..01bcfb0dddae06 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -163,8 +163,7 @@ directly from the command line using the :option:`-m` flag. For example:: This command will cause ``__main__.py`` to run. For more details about the :option:`-m` flag, see :mod:`runpy`. How you utilize this mechanism will depend on the nature of the package you are writing, but in this hypothetical case, it -might make sense to allow the teacher to search for students using -:mod:`argparse`:: +might make sense to allow the teacher to search for students:: # bandclass/__main__.py @@ -172,7 +171,7 @@ might make sense to allow the teacher to search for students using from .student import search_students student_name = sys.argv[2] if len(sys.argv) >= 2 else '' - print('Found student: {search_students(student_name)}') + print(f'Found student: {search_students(student_name)}') Note that ``from .student import search_students`` is an example of a relative import. This import style must be used when referencing modules within a From 757b03ad303eb63212bafc01b9104aa815b2d7c8 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 10 Aug 2021 13:08:28 -0400 Subject: [PATCH 27/46] incorporate suggested changes from @Fidget-Spinner --- Doc/library/__main__.rst | 6 +++--- .../Documentation/2021-06-23-15-21-36.bpo-39452.o_I-6d.rst | 3 ++- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 01bcfb0dddae06..7f4a3cbe62a768 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -9,10 +9,10 @@ In Python, ``__main__`` is not a single mechanism in the language, but in fact is part of two quite different constructs: -1. The ``__name__ == '__main__'`` statement +1. The ``__name__ == '__main__'`` expression 2. The ``__main__.py`` file in Python packages -Each of these mechanisms are related to Python modules; how users interact with +Both of these mechanisms are related to Python modules; how users interact with them and how they interact with each other. See section :ref:`tut-modules`. @@ -84,7 +84,7 @@ For detailed documentation on Python packaging, see the *main* functions are often used to create command line tools by specifying them as entry points for console scripts. When this is done, pip inserts the function call into a template script, where the return value of *main* is -passed into sys.exit. For example:: +passed into :func:`sys.exit`. For example:: sys.exit(main()) diff --git a/Misc/NEWS.d/next/Documentation/2021-06-23-15-21-36.bpo-39452.o_I-6d.rst b/Misc/NEWS.d/next/Documentation/2021-06-23-15-21-36.bpo-39452.o_I-6d.rst index c7c0b510647db0..5c8cbd8e652232 100644 --- a/Misc/NEWS.d/next/Documentation/2021-06-23-15-21-36.bpo-39452.o_I-6d.rst +++ b/Misc/NEWS.d/next/Documentation/2021-06-23-15-21-36.bpo-39452.o_I-6d.rst @@ -1,3 +1,4 @@ Rewrote ``Doc/library/__main__.rst``. Broadened scope of the document to explicitly discuss and differentiate between ``__main__.py`` in packages -versus the ``__name__ == '__main__'`` statement. +versus the ``__name__ == '__main__'`` expression (and the idioms that +surround it). From 8e864680d1f9bdb7aa99043ce612fcccfae2e650 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 10 Aug 2021 13:36:36 -0400 Subject: [PATCH 28/46] incorporate suggested changes from @yaseppochi --- Doc/library/__main__.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 7f4a3cbe62a768..a83b4397a680b2 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -71,8 +71,9 @@ encapsulates the program's primary behavior:: sys.exit(main()) # next section explains the use of sys.exit This has the added benefit of the *echo* function itself being isolated and -importable elsewhere. None of the code in ``echo.py`` will execute at -import-time. +importable elsewhere. When ``echo.py`` is imported, the ``echo`` and ``main`` +functions will be defined, but neither of them will be called, because +``__name__ != '__main__'``. Packaging Considerations From f33a081500ff0f994762f70d0bd05a4c6265351b Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Tue, 10 Aug 2021 13:44:53 -0400 Subject: [PATCH 29/46] fix formatting --- Doc/library/__main__.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index a83b4397a680b2..0d104186984506 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -77,7 +77,7 @@ functions will be defined, but neither of them will be called, because Packaging Considerations -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^ For detailed documentation on Python packaging, see the `Python Packaging User Guide. `_ From 14bad859f929df71a7aede42ff6a2be9dd3ccef6 Mon Sep 17 00:00:00 2001 From: Jack DeVries <58614260+jdevries3133@users.noreply.github.com> Date: Thu, 12 Aug 2021 10:11:05 -0400 Subject: [PATCH 30/46] name equals main MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Éric Araujo --- Doc/library/__main__.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 0d104186984506..7cd637da4e31b6 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -16,7 +16,7 @@ Both of these mechanisms are related to Python modules; how users interact with them and how they interact with each other. See section :ref:`tut-modules`. -.. _name_is_main: +.. _name_equals_main: ``__name__ == '__main__'`` --------------------------- From b9db705f47e7ed84619255410a36d4c0bda19575 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Thu, 12 Aug 2021 10:16:58 -0400 Subject: [PATCH 31/46] also change reference to name equals main section --- Doc/library/__main__.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 7cd637da4e31b6..f0bb7ba83607fd 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -105,7 +105,7 @@ convention. ------------------- All the values in the ``__main__`` namespace can be imported elsewhere in -Python packages. See section :ref:`name_is_main` for a list of where the +Python packages. See section :ref:`name_equals_main` for a list of where the ``__main__`` package is in different Python execution scenarios. Here is an example package that consumes the ``__main__`` namespace:: From 168c77457e962b79a431b0a5b855148a6a3b505d Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Thu, 12 Aug 2021 11:00:53 -0400 Subject: [PATCH 32/46] implement feedback from @holdenweb, python-dev, and @merwork --- Doc/library/__main__.rst | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index f0bb7ba83607fd..4454b6d706fa09 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -21,16 +21,18 @@ them and how they interact with each other. See section :ref:`tut-modules`. ``__name__ == '__main__'`` --------------------------- -``'__main__'`` is the name of the environment where top-level code is run. +When a Python module or package is imported, ``__name__`` is set to the +module's name. Usually, this is the name of the Python file itself without the +``.py`` extension. For a more detailed look at how ``__name__`` is set in +all situations, see section :ref:`tut-modules`. + +In some circumstances, ``__name__`` is set to the string ``'__main__'``. +``__main__`` is the name of the environment where top-level code is run. "Top-level code" means when a Python module is initialized from an interactive -prompt, from standard input, from a file argument, from a :option:`-c` -argument, or from a :option:`-m` argument, but **not** when it is initialized +prompt, from standard input, from a file argument, with the :option:`-c` +argument, or with the :option:`-m` argument, but **not** when it is initialized from an import statement. In any of these situations, the module's -``__name__`` is set equal to ``'__main__'``. - -The only other context in which Python code is run is when it is imported -through an import statement. In that case, ``__name__`` is set equal to the -module's name: usually the name of the file without the ``.py`` extension. +``__name__`` is set to ``'__main__'``. As a result, a module can discover whether or not it is running in the top-level environment by checking its own ``__name__``, which allows a common @@ -92,13 +94,16 @@ passed into :func:`sys.exit`. For example:: Since the call to *main* is wrapped in :func:`sys.exit`, the expectation is that your function will return some value acceptable as an input to :func:`sys.exit`; typically, an integer or ``None`` (which is implicitly -returned if your function does not have a return statement). +returned if your function does not have a return statement). By proactively following this convention ourselves, our module will have the -same behavior when run directly (``python3 echo.py``) as it will have if we -later package it as a console script entry-point in a pip-installable package. -That is why the ``echo.py`` example from earlier used the ``sys.exit(main())`` -convention. +same behavior when run directly (i.e. ``python3 echo.py``) as it will have if +we later package it as a console script entry-point in a pip-installable +package. In particular, be careful about returning strings from your *main* +function. :func:`sys.exit` will interpret a string as a failure message, so +your program will have an exit code of ``1``, indicating failure, and the +string will be written to :data:`sys.stderr`. The ``echo.py`` example from +earlier exemplifies using the ``sys.exit(main())`` convention. ``import __main__`` From c450171d0bc45e176379e8e2f033dc42b2ba52d7 Mon Sep 17 00:00:00 2001 From: Jack DeVries Date: Thu, 12 Aug 2021 17:30:08 -0400 Subject: [PATCH 33/46] fix trailing whitespace --- Doc/library/__main__.rst | 2 +- Doc/library/__main__.rst.bak | 188 +++++++++++++++++++++++++++++++++++ 2 files changed, 189 insertions(+), 1 deletion(-) create mode 100644 Doc/library/__main__.rst.bak diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 4454b6d706fa09..52e76d2782382c 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -94,7 +94,7 @@ passed into :func:`sys.exit`. For example:: Since the call to *main* is wrapped in :func:`sys.exit`, the expectation is that your function will return some value acceptable as an input to :func:`sys.exit`; typically, an integer or ``None`` (which is implicitly -returned if your function does not have a return statement). +returned if your function does not have a return statement). By proactively following this convention ourselves, our module will have the same behavior when run directly (i.e. ``python3 echo.py``) as it will have if diff --git a/Doc/library/__main__.rst.bak b/Doc/library/__main__.rst.bak new file mode 100644 index 00000000000000..4454b6d706fa09 --- /dev/null +++ b/Doc/library/__main__.rst.bak @@ -0,0 +1,188 @@ +:mod:`__main__` --- CLIs, import-time behavior, and ``__name__ == '__main__'`` +============================================================================== + +.. module:: __main__ + :synopsis: Command line interfaces, import-time behavior, and ``__name__ == '__main__'`` + +-------------- + +In Python, ``__main__`` is not a single mechanism in the language, but in fact +is part of two quite different constructs: + +1. The ``__name__ == '__main__'`` expression +2. The ``__main__.py`` file in Python packages + +Both of these mechanisms are related to Python modules; how users interact with +them and how they interact with each other. See section :ref:`tut-modules`. + + +.. _name_equals_main: + +``__name__ == '__main__'`` +--------------------------- + +When a Python module or package is imported, ``__name__`` is set to the +module's name. Usually, this is the name of the Python file itself without the +``.py`` extension. For a more detailed look at how ``__name__`` is set in +all situations, see section :ref:`tut-modules`. + +In some circumstances, ``__name__`` is set to the string ``'__main__'``. +``__main__`` is the name of the environment where top-level code is run. +"Top-level code" means when a Python module is initialized from an interactive +prompt, from standard input, from a file argument, with the :option:`-c` +argument, or with the :option:`-m` argument, but **not** when it is initialized +from an import statement. In any of these situations, the module's +``__name__`` is set to ``'__main__'``. + +As a result, a module can discover whether or not it is running in the +top-level environment by checking its own ``__name__``, which allows a common +idiom for conditionally executing code when the module is not initialized from +an import statement:: + + if __name__ == '__main__': + # Execute when the module is not initialized from an import statement. + ... + + +Idiomatic Usage +^^^^^^^^^^^^^^^ + +Putting as few statements as possible in the block below ``if __name___ == +'__main__'`` can improve code clarity. Most often, a function named *main* +encapsulates the program's primary behavior:: + + # echo.py + + import shlex + import sys + + + def echo(phrase: str): + """A dummy wrapper around print.""" + # for demonstration purposes, you can imagine that there is some + # valuable and reusable logic inside this function + print(phrase) + + + def main(): + """Echo the input arguments to standard output""" + echo(shlex.join(sys.argv)) + + + if __name__ == '__main__': + sys.exit(main()) # next section explains the use of sys.exit + +This has the added benefit of the *echo* function itself being isolated and +importable elsewhere. When ``echo.py`` is imported, the ``echo`` and ``main`` +functions will be defined, but neither of them will be called, because +``__name__ != '__main__'``. + + +Packaging Considerations +^^^^^^^^^^^^^^^^^^^^^^^^ + +For detailed documentation on Python packaging, see the +`Python Packaging User Guide. `_ + +*main* functions are often used to create command line tools by specifying them +as entry points for console scripts. When this is done, pip inserts the +function call into a template script, where the return value of *main* is +passed into :func:`sys.exit`. For example:: + + sys.exit(main()) + +Since the call to *main* is wrapped in :func:`sys.exit`, the expectation is +that your function will return some value acceptable as an input to +:func:`sys.exit`; typically, an integer or ``None`` (which is implicitly +returned if your function does not have a return statement). + +By proactively following this convention ourselves, our module will have the +same behavior when run directly (i.e. ``python3 echo.py``) as it will have if +we later package it as a console script entry-point in a pip-installable +package. In particular, be careful about returning strings from your *main* +function. :func:`sys.exit` will interpret a string as a failure message, so +your program will have an exit code of ``1``, indicating failure, and the +string will be written to :data:`sys.stderr`. The ``echo.py`` example from +earlier exemplifies using the ``sys.exit(main())`` convention. + + +``import __main__`` +------------------- + +All the values in the ``__main__`` namespace can be imported elsewhere in +Python packages. See section :ref:`name_equals_main` for a list of where the +``__main__`` package is in different Python execution scenarios. + +Here is an example package that consumes the ``__main__`` namespace:: + + # namely.py + + import __main__ + + def did_user_define_their_name(): + return 'my_name' in dir(__main__) + + def print_user_name(): + if did_user_define_their_name(): + print(__main__.my_name) + else: + print('Tell us your name by defining the variable `my_name`!') + +The Python REPL is one example of a "top-level environment", so anything +defined in the REPL becomes part of the ``__main__`` package:: + + >>> import namely + >>> namely.did_user_define_their_name() + False + >>> namely.print_user_name() + Tell us your name by defining the variable `my_name`! + >>> my_name = 'David' + >>> namely.did_user_define_their_name() + True + >>> namely.print_user_name() + David + +The ``__main__`` package is used in the implementation of :mod:`pdb` and +:mod:`rlcompleter`. + + +``__main__.py`` in Python Packages +---------------------------------- + +If you are not familiar with Python packages, see section :ref:`tut-packages`. +Most commonly, the ``__main__.py`` file is used to provide a command line +interface for a package. Consider the following hypothetical package, +"bandclass": + +.. code-block:: text + + bandclass + ├── __init__.py + ├── __main__.py + └── student.py + +``__main__.py`` will be executed when the package itself is invoked +directly from the command line using the :option:`-m` flag. For example:: + + python3 -m bandclass + +This command will cause ``__main__.py`` to run. For more details about the +:option:`-m` flag, see :mod:`runpy`. How you utilize this mechanism will depend +on the nature of the package you are writing, but in this hypothetical case, it +might make sense to allow the teacher to search for students:: + + # bandclass/__main__.py + + import sys + from .student import search_students + + student_name = sys.argv[2] if len(sys.argv) >= 2 else '' + print(f'Found student: {search_students(student_name)}') + +Note that ``from .student import search_students`` is an example of a relative +import. This import style must be used when referencing modules within a +package. For more details, see :ref:`tut-modules`; or, more specifically, +:ref:`intra-package-references`. + +For an example of a package using ``__main__.py`` in our standard library, see +:mod:`venv`, and its invocation via ``python3 -m venv [directory]``. From 077e7a4b4d2fd9c772dbe0f84279cabfe670bf78 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 12:00:27 +0200 Subject: [PATCH 34/46] Remove .bak file --- Doc/library/__main__.rst.bak | 188 ----------------------------------- 1 file changed, 188 deletions(-) delete mode 100644 Doc/library/__main__.rst.bak diff --git a/Doc/library/__main__.rst.bak b/Doc/library/__main__.rst.bak deleted file mode 100644 index 4454b6d706fa09..00000000000000 --- a/Doc/library/__main__.rst.bak +++ /dev/null @@ -1,188 +0,0 @@ -:mod:`__main__` --- CLIs, import-time behavior, and ``__name__ == '__main__'`` -============================================================================== - -.. module:: __main__ - :synopsis: Command line interfaces, import-time behavior, and ``__name__ == '__main__'`` - --------------- - -In Python, ``__main__`` is not a single mechanism in the language, but in fact -is part of two quite different constructs: - -1. The ``__name__ == '__main__'`` expression -2. The ``__main__.py`` file in Python packages - -Both of these mechanisms are related to Python modules; how users interact with -them and how they interact with each other. See section :ref:`tut-modules`. - - -.. _name_equals_main: - -``__name__ == '__main__'`` ---------------------------- - -When a Python module or package is imported, ``__name__`` is set to the -module's name. Usually, this is the name of the Python file itself without the -``.py`` extension. For a more detailed look at how ``__name__`` is set in -all situations, see section :ref:`tut-modules`. - -In some circumstances, ``__name__`` is set to the string ``'__main__'``. -``__main__`` is the name of the environment where top-level code is run. -"Top-level code" means when a Python module is initialized from an interactive -prompt, from standard input, from a file argument, with the :option:`-c` -argument, or with the :option:`-m` argument, but **not** when it is initialized -from an import statement. In any of these situations, the module's -``__name__`` is set to ``'__main__'``. - -As a result, a module can discover whether or not it is running in the -top-level environment by checking its own ``__name__``, which allows a common -idiom for conditionally executing code when the module is not initialized from -an import statement:: - - if __name__ == '__main__': - # Execute when the module is not initialized from an import statement. - ... - - -Idiomatic Usage -^^^^^^^^^^^^^^^ - -Putting as few statements as possible in the block below ``if __name___ == -'__main__'`` can improve code clarity. Most often, a function named *main* -encapsulates the program's primary behavior:: - - # echo.py - - import shlex - import sys - - - def echo(phrase: str): - """A dummy wrapper around print.""" - # for demonstration purposes, you can imagine that there is some - # valuable and reusable logic inside this function - print(phrase) - - - def main(): - """Echo the input arguments to standard output""" - echo(shlex.join(sys.argv)) - - - if __name__ == '__main__': - sys.exit(main()) # next section explains the use of sys.exit - -This has the added benefit of the *echo* function itself being isolated and -importable elsewhere. When ``echo.py`` is imported, the ``echo`` and ``main`` -functions will be defined, but neither of them will be called, because -``__name__ != '__main__'``. - - -Packaging Considerations -^^^^^^^^^^^^^^^^^^^^^^^^ - -For detailed documentation on Python packaging, see the -`Python Packaging User Guide. `_ - -*main* functions are often used to create command line tools by specifying them -as entry points for console scripts. When this is done, pip inserts the -function call into a template script, where the return value of *main* is -passed into :func:`sys.exit`. For example:: - - sys.exit(main()) - -Since the call to *main* is wrapped in :func:`sys.exit`, the expectation is -that your function will return some value acceptable as an input to -:func:`sys.exit`; typically, an integer or ``None`` (which is implicitly -returned if your function does not have a return statement). - -By proactively following this convention ourselves, our module will have the -same behavior when run directly (i.e. ``python3 echo.py``) as it will have if -we later package it as a console script entry-point in a pip-installable -package. In particular, be careful about returning strings from your *main* -function. :func:`sys.exit` will interpret a string as a failure message, so -your program will have an exit code of ``1``, indicating failure, and the -string will be written to :data:`sys.stderr`. The ``echo.py`` example from -earlier exemplifies using the ``sys.exit(main())`` convention. - - -``import __main__`` -------------------- - -All the values in the ``__main__`` namespace can be imported elsewhere in -Python packages. See section :ref:`name_equals_main` for a list of where the -``__main__`` package is in different Python execution scenarios. - -Here is an example package that consumes the ``__main__`` namespace:: - - # namely.py - - import __main__ - - def did_user_define_their_name(): - return 'my_name' in dir(__main__) - - def print_user_name(): - if did_user_define_their_name(): - print(__main__.my_name) - else: - print('Tell us your name by defining the variable `my_name`!') - -The Python REPL is one example of a "top-level environment", so anything -defined in the REPL becomes part of the ``__main__`` package:: - - >>> import namely - >>> namely.did_user_define_their_name() - False - >>> namely.print_user_name() - Tell us your name by defining the variable `my_name`! - >>> my_name = 'David' - >>> namely.did_user_define_their_name() - True - >>> namely.print_user_name() - David - -The ``__main__`` package is used in the implementation of :mod:`pdb` and -:mod:`rlcompleter`. - - -``__main__.py`` in Python Packages ----------------------------------- - -If you are not familiar with Python packages, see section :ref:`tut-packages`. -Most commonly, the ``__main__.py`` file is used to provide a command line -interface for a package. Consider the following hypothetical package, -"bandclass": - -.. code-block:: text - - bandclass - ├── __init__.py - ├── __main__.py - └── student.py - -``__main__.py`` will be executed when the package itself is invoked -directly from the command line using the :option:`-m` flag. For example:: - - python3 -m bandclass - -This command will cause ``__main__.py`` to run. For more details about the -:option:`-m` flag, see :mod:`runpy`. How you utilize this mechanism will depend -on the nature of the package you are writing, but in this hypothetical case, it -might make sense to allow the teacher to search for students:: - - # bandclass/__main__.py - - import sys - from .student import search_students - - student_name = sys.argv[2] if len(sys.argv) >= 2 else '' - print(f'Found student: {search_students(student_name)}') - -Note that ``from .student import search_students`` is an example of a relative -import. This import style must be used when referencing modules within a -package. For more details, see :ref:`tut-modules`; or, more specifically, -:ref:`intra-package-references`. - -For an example of a package using ``__main__.py`` in our standard library, see -:mod:`venv`, and its invocation via ``python3 -m venv [directory]``. From eb42489fc87454b93f89eb1f431056ffa7a0bb74 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 13:26:10 +0200 Subject: [PATCH 35/46] Thorough editing pass --- Doc/library/__main__.rst | 180 +++++++++++++++++++++++++++++---------- 1 file changed, 136 insertions(+), 44 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 52e76d2782382c..223d0186169a92 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -6,14 +6,16 @@ -------------- -In Python, ``__main__`` is not a single mechanism in the language, but in fact -is part of two quite different constructs: +In Python, the special name ``__main__`` is used for two important constructs: -1. The ``__name__ == '__main__'`` expression -2. The ``__main__.py`` file in Python packages +1. the name of the top-level environment of the program, which can be + checked using the ``__name__ == '__main__'`` expression; and +2. the ``__main__.py`` file in Python packages. Both of these mechanisms are related to Python modules; how users interact with -them and how they interact with each other. See section :ref:`tut-modules`. +them and how they interact with each other. They are explained in detail +below. If you're new to Python modules, see the tutorial section +:ref:`tut-modules` for an introduction. .. _name_equals_main: @@ -23,8 +25,21 @@ them and how they interact with each other. See section :ref:`tut-modules`. When a Python module or package is imported, ``__name__`` is set to the module's name. Usually, this is the name of the Python file itself without the -``.py`` extension. For a more detailed look at how ``__name__`` is set in -all situations, see section :ref:`tut-modules`. +``.py`` extension:: + + >>> import configparser + >>> configparser.__name__ + 'configparser' + +If the file is part of a package, ``__name__`` will also include the parent +package's path:: + + >>> from concurrent.futures import process + >>> process.__name__ + 'concurrent.futures.process' + +For a more detailed look at how ``__name__`` is set in all situations, see the +tutorial section :ref:`tut-modules`. In some circumstances, ``__name__`` is set to the string ``'__main__'``. ``__main__`` is the name of the environment where top-level code is run. @@ -47,35 +62,50 @@ an import statement:: Idiomatic Usage ^^^^^^^^^^^^^^^ +Some modules contain code which is intended for script use only, like parsing +command-line arguments or fetching data from standard input. When a module +like this were to be imported from a different module, for example to unit test +it, the script code would unintentionally execute as well. + +This is where using the ``if __name__ == '__main__'`` code block comes in +handy. Code within this block won't run unless the module is executed in the +top-level environment. + Putting as few statements as possible in the block below ``if __name___ == -'__main__'`` can improve code clarity. Most often, a function named *main* -encapsulates the program's primary behavior:: +'__main__'`` can improve code clarity and correctness. Most often, a function +named ``main`` encapsulates the program's primary behavior:: # echo.py import shlex import sys - - def echo(phrase: str): + def echo(phrase: str) -> None: """A dummy wrapper around print.""" # for demonstration purposes, you can imagine that there is some # valuable and reusable logic inside this function print(phrase) - - def main(): + def main() -> int: """Echo the input arguments to standard output""" - echo(shlex.join(sys.argv)) - + phrase = shlex.join(sys.argv) + echo(phrase) + return 0 if __name__ == '__main__': sys.exit(main()) # next section explains the use of sys.exit -This has the added benefit of the *echo* function itself being isolated and -importable elsewhere. When ``echo.py`` is imported, the ``echo`` and ``main`` -functions will be defined, but neither of them will be called, because -``__name__ != '__main__'``. +Note that if the module didn't encapsulate code inside the ``main`` function +but instead put it directly within the ``if __name__ == '__main__'`` block, +the ``phrase`` variable would be global to the entire module. This is +error-prone as other functions within the module could be unintentionally using +the global variable instead of a local name. A ``main`` function solves this +problem. + +Using a ``main`` function has the added benefit of the ``echo`` function itself +being isolated and importable elsewhere. When ``echo.py`` is imported, the +``echo`` and ``main`` functions will be defined, but neither of them will be +called, because ``__name__ != '__main__'``. Packaging Considerations @@ -84,14 +114,15 @@ Packaging Considerations For detailed documentation on Python packaging, see the `Python Packaging User Guide. `_ -*main* functions are often used to create command line tools by specifying them -as entry points for console scripts. When this is done, pip inserts the -function call into a template script, where the return value of *main* is -passed into :func:`sys.exit`. For example:: +``main`` functions are often used to create command line tools by specifying +them as entry points for console scripts. When this is done, +`pip `_ inserts the function call into a template script, +where the return value of ``main`` is passed into :func:`sys.exit`. +For example:: sys.exit(main()) -Since the call to *main* is wrapped in :func:`sys.exit`, the expectation is +Since the call to ``main`` is wrapped in :func:`sys.exit`, the expectation is that your function will return some value acceptable as an input to :func:`sys.exit`; typically, an integer or ``None`` (which is implicitly returned if your function does not have a return statement). @@ -99,8 +130,10 @@ returned if your function does not have a return statement). By proactively following this convention ourselves, our module will have the same behavior when run directly (i.e. ``python3 echo.py``) as it will have if we later package it as a console script entry-point in a pip-installable -package. In particular, be careful about returning strings from your *main* -function. :func:`sys.exit` will interpret a string as a failure message, so +package. + +In particular, be careful about returning strings from your ``main`` function. +:func:`sys.exit` will interpret a string argument as a failure message, so your program will have an exit code of ``1``, indicating failure, and the string will be written to :data:`sys.stderr`. The ``echo.py`` example from earlier exemplifies using the ``sys.exit(main())`` convention. @@ -109,11 +142,11 @@ earlier exemplifies using the ``sys.exit(main())`` convention. ``import __main__`` ------------------- -All the values in the ``__main__`` namespace can be imported elsewhere in -Python packages. See section :ref:`name_equals_main` for a list of where the -``__main__`` package is in different Python execution scenarios. +Regardless of which module a Python program was started with, other modules +running within that same program can import the top-level environment's scope +(:term:`namespace`) by importing the ``__main__`` module. -Here is an example package that consumes the ``__main__`` namespace:: +Here is an example module that consumes the ``__main__`` namespace:: # namely.py @@ -123,36 +156,85 @@ Here is an example package that consumes the ``__main__`` namespace:: return 'my_name' in dir(__main__) def print_user_name(): - if did_user_define_their_name(): - print(__main__.my_name) + if not did_user_define_their_name(): + raise ValueError('Define the variable `my_name`!') + + if '__file__' in dir(__main__): + print(__main__.my_name, "found in file", __main__.__file__) else: - print('Tell us your name by defining the variable `my_name`!') + print(__main__.my_name) + +Example usage of this module could be as follows:: + + # start.py + + import sys + + from namely import print_user_name + + # my_name = "Dinsdale" + + def main(): + try: + print_user_name() + except ValueError as ve: + return str(ve) + + if __name__ == "__main__": + sys.exit(main()) + +Now, if we started our program, the result would look like this: + +.. code-block:: text + + $ python3 start.py + Define the variable `my_name`! + +The exit code of the program would be 1, indicating an error. Uncommenting the +line with ``my_name = "Dinsdale"`` fixes the program and now it exits with +status code 0, indicating success: + +.. code-block:: text + + $ python3 start.py + Dinsdale found in file /path/to/start.py + +Note that importing ``__main__`` doesn't cause any issues with unintentionally +running top-level code meant for script use which is put in the +``if __name__ == "__main__"`` block of ``start.py``. It's because this code +has already executed at the start of the program and the module is already +fully loaded when ``namely.py`` imports ``__main__``. The Python REPL is one example of a "top-level environment", so anything -defined in the REPL becomes part of the ``__main__`` package:: +defined in the REPL becomes part of the ``__main__`` scope:: >>> import namely >>> namely.did_user_define_their_name() False >>> namely.print_user_name() - Tell us your name by defining the variable `my_name`! - >>> my_name = 'David' + Traceback (most recent call last): + ... + ValueError: Define the variable `my_name`! + >>> my_name = 'Jabberwocky' >>> namely.did_user_define_their_name() True >>> namely.print_user_name() - David + Jabberwocky + +Note that in this case the ``__main__`` scope doesn't contain a ``__file__`` +attribute as it's interactive. -The ``__main__`` package is used in the implementation of :mod:`pdb` and +The ``__main__`` scope is used in the implementation of :mod:`pdb` and :mod:`rlcompleter`. ``__main__.py`` in Python Packages ---------------------------------- -If you are not familiar with Python packages, see section :ref:`tut-packages`. -Most commonly, the ``__main__.py`` file is used to provide a command line -interface for a package. Consider the following hypothetical package, -"bandclass": +If you are not familiar with Python packages, see section :ref:`tut-packages` +of the tutorial. Most commonly, the ``__main__.py`` file is used to provide +a command line interface for a package. Consider the following hypothetical +package, "bandclass": .. code-block:: text @@ -181,8 +263,18 @@ might make sense to allow the teacher to search for students:: Note that ``from .student import search_students`` is an example of a relative import. This import style must be used when referencing modules within a -package. For more details, see :ref:`tut-modules`; or, more specifically, -:ref:`intra-package-references`. +package. For more details, see :ref:`intra-package-references` in the +:ref:`tut-modules` section of the tutorial. For an example of a package using ``__main__.py`` in our standard library, see :mod:`venv`, and its invocation via ``python3 -m venv [directory]``. + +Idiomatic Usage +^^^^^^^^^^^^^^^ + +Note that ``__main__.py`` files in Python packages cannot use the +``if __name__ == '__main__'`` idiom because the name of the module is *always* +``'__main__'``. This is why the example above doesn't use the block. More +importantly, this is why ``__main__.py`` files should be minimal, importing +functions to execute from other modules. Those other modules can then be +easily unit-tested, and are properly reusable. From 7c61e7877b59a9cbc98c1183103218c7545685db Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 13:43:38 +0200 Subject: [PATCH 36/46] Move `__main__.py` section above the `import __main__` section --- Doc/library/__main__.rst | 108 ++++++++++++++++++++------------------- 1 file changed, 55 insertions(+), 53 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 223d0186169a92..69a69e575d49ac 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -139,12 +139,66 @@ string will be written to :data:`sys.stderr`. The ``echo.py`` example from earlier exemplifies using the ``sys.exit(main())`` convention. +``__main__.py`` in Python Packages +---------------------------------- + +If you are not familiar with Python packages, see section :ref:`tut-packages` +of the tutorial. Most commonly, the ``__main__.py`` file is used to provide +a command line interface for a package. Consider the following hypothetical +package, "bandclass": + +.. code-block:: text + + bandclass + ├── __init__.py + ├── __main__.py + └── student.py + +``__main__.py`` will be executed when the package itself is invoked +directly from the command line using the :option:`-m` flag. For example:: + + python3 -m bandclass + +This command will cause ``__main__.py`` to run. For more details about the +:option:`-m` flag, see :mod:`runpy`. How you utilize this mechanism will depend +on the nature of the package you are writing, but in this hypothetical case, it +might make sense to allow the teacher to search for students:: + + # bandclass/__main__.py + + import sys + from .student import search_students + + student_name = sys.argv[2] if len(sys.argv) >= 2 else '' + print(f'Found student: {search_students(student_name)}') + +Note that ``from .student import search_students`` is an example of a relative +import. This import style must be used when referencing modules within a +package. For more details, see :ref:`intra-package-references` in the +:ref:`tut-modules` section of the tutorial. + +For an example of a package using ``__main__.py`` in our standard library, see +:mod:`venv`, and its invocation via ``python3 -m venv [directory]``. + +Idiomatic Usage +^^^^^^^^^^^^^^^ + +Note that ``__main__.py`` files in Python packages cannot use the +``if __name__ == '__main__'`` idiom because the name of the module is *always* +``'__main__'``. This is why the example above doesn't use the block. More +importantly, this is why ``__main__.py`` files should be minimal, importing +functions to execute from other modules. Those other modules can then be +easily unit-tested, and are properly reusable. + + ``import __main__`` ------------------- Regardless of which module a Python program was started with, other modules running within that same program can import the top-level environment's scope -(:term:`namespace`) by importing the ``__main__`` module. +(:term:`namespace`) by importing the ``__main__`` module. This doesn't import +a ``__main__.py`` file but rather whichever module that received the special +name ``'__main__'``. Here is an example module that consumes the ``__main__`` namespace:: @@ -226,55 +280,3 @@ attribute as it's interactive. The ``__main__`` scope is used in the implementation of :mod:`pdb` and :mod:`rlcompleter`. - - -``__main__.py`` in Python Packages ----------------------------------- - -If you are not familiar with Python packages, see section :ref:`tut-packages` -of the tutorial. Most commonly, the ``__main__.py`` file is used to provide -a command line interface for a package. Consider the following hypothetical -package, "bandclass": - -.. code-block:: text - - bandclass - ├── __init__.py - ├── __main__.py - └── student.py - -``__main__.py`` will be executed when the package itself is invoked -directly from the command line using the :option:`-m` flag. For example:: - - python3 -m bandclass - -This command will cause ``__main__.py`` to run. For more details about the -:option:`-m` flag, see :mod:`runpy`. How you utilize this mechanism will depend -on the nature of the package you are writing, but in this hypothetical case, it -might make sense to allow the teacher to search for students:: - - # bandclass/__main__.py - - import sys - from .student import search_students - - student_name = sys.argv[2] if len(sys.argv) >= 2 else '' - print(f'Found student: {search_students(student_name)}') - -Note that ``from .student import search_students`` is an example of a relative -import. This import style must be used when referencing modules within a -package. For more details, see :ref:`intra-package-references` in the -:ref:`tut-modules` section of the tutorial. - -For an example of a package using ``__main__.py`` in our standard library, see -:mod:`venv`, and its invocation via ``python3 -m venv [directory]``. - -Idiomatic Usage -^^^^^^^^^^^^^^^ - -Note that ``__main__.py`` files in Python packages cannot use the -``if __name__ == '__main__'`` idiom because the name of the module is *always* -``'__main__'``. This is why the example above doesn't use the block. More -importantly, this is why ``__main__.py`` files should be minimal, importing -functions to execute from other modules. Those other modules can then be -easily unit-tested, and are properly reusable. From 45a9425cba3d8e0591d71533d57e80e875f9ccb6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 14:05:34 +0200 Subject: [PATCH 37/46] s/command line/command-line/ --- Doc/library/__main__.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 69a69e575d49ac..97a7d10e219dc1 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -2,7 +2,7 @@ ============================================================================== .. module:: __main__ - :synopsis: Command line interfaces, import-time behavior, and ``__name__ == '__main__'`` + :synopsis: Command-line interfaces, import-time behavior, and ``__name__ == '__main__'`` -------------- @@ -114,7 +114,7 @@ Packaging Considerations For detailed documentation on Python packaging, see the `Python Packaging User Guide. `_ -``main`` functions are often used to create command line tools by specifying +``main`` functions are often used to create command-line tools by specifying them as entry points for console scripts. When this is done, `pip `_ inserts the function call into a template script, where the return value of ``main`` is passed into :func:`sys.exit`. @@ -144,7 +144,7 @@ earlier exemplifies using the ``sys.exit(main())`` convention. If you are not familiar with Python packages, see section :ref:`tut-packages` of the tutorial. Most commonly, the ``__main__.py`` file is used to provide -a command line interface for a package. Consider the following hypothetical +a command-line interface for a package. Consider the following hypothetical package, "bandclass": .. code-block:: text From 073e9d72347b28753fe40ec536c3aec3e20d195a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 14:15:08 +0200 Subject: [PATCH 38/46] Mention asyncio.__main__ --- Doc/library/__main__.rst | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 97a7d10e219dc1..fb196cb82b6235 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -62,7 +62,7 @@ an import statement:: Idiomatic Usage ^^^^^^^^^^^^^^^ -Some modules contain code which is intended for script use only, like parsing +Some modules contain code that is intended for script use only, like parsing command-line arguments or fetching data from standard input. When a module like this were to be imported from a different module, for example to unit test it, the script code would unintentionally execute as well. @@ -183,12 +183,21 @@ For an example of a package using ``__main__.py`` in our standard library, see Idiomatic Usage ^^^^^^^^^^^^^^^ -Note that ``__main__.py`` files in Python packages cannot use the -``if __name__ == '__main__'`` idiom because the name of the module is *always* -``'__main__'``. This is why the example above doesn't use the block. More -importantly, this is why ``__main__.py`` files should be minimal, importing +The contents of ``__main__.py`` typically isn't fenced with +``if __name__ == '__main__'`` blocks. Instead, those files are kept short, functions to execute from other modules. Those other modules can then be -easily unit-tested, and are properly reusable. +easily unit-tested and are properly reusable. + +If used, an ``if __name__ == '__main__'`` block will still work as expected +for a ``__main__.py`` file within a package, because its ``__name__`` +attribute will include the package's path if imported:: + + >>> import asyncio.__main__ + >>> asyncio.__main__.__name__ + 'asyncio.__main__' + +That being said, minimal ``__main__.py`` like the :mod:`venv` one mentioned +above are preferred. ``import __main__`` From ccb90049bca36085850797f80f61b5659c9e2a6e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 14:44:25 +0200 Subject: [PATCH 39/46] Restore proper document name --- Doc/library/__main__.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index fb196cb82b6235..71eec7c5702178 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -1,8 +1,9 @@ -:mod:`__main__` --- CLIs, import-time behavior, and ``__name__ == '__main__'`` -============================================================================== +:mod:`__main__` --- Top-level code environment +============================================== .. module:: __main__ - :synopsis: Command-line interfaces, import-time behavior, and ``__name__ == '__main__'`` + :synopsis: The environment where top-level code is run. Covers command-line + interfaces, import-time behavior, and ``__name__ == '__main__'``. -------------- From f8630faef7a958de19bf0030de6d4c43f90f1935 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 15:05:36 +0200 Subject: [PATCH 40/46] Use proper .. seealso:: sections. --- Doc/library/__main__.rst | 47 +++++++++++++++++++++++++++------------- 1 file changed, 32 insertions(+), 15 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 71eec7c5702178..5dbb3f37fcf7cc 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -39,9 +39,6 @@ package's path:: >>> process.__name__ 'concurrent.futures.process' -For a more detailed look at how ``__name__`` is set in all situations, see the -tutorial section :ref:`tut-modules`. - In some circumstances, ``__name__`` is set to the string ``'__main__'``. ``__main__`` is the name of the environment where top-level code is run. "Top-level code" means when a Python module is initialized from an interactive @@ -59,6 +56,11 @@ an import statement:: # Execute when the module is not initialized from an import statement. ... +.. seealso:: + + For a more detailed look at how ``__name__`` is set in all situations, see + the tutorial section :ref:`tut-modules`. + Idiomatic Usage ^^^^^^^^^^^^^^^ @@ -112,9 +114,6 @@ called, because ``__name__ != '__main__'``. Packaging Considerations ^^^^^^^^^^^^^^^^^^^^^^^^ -For detailed documentation on Python packaging, see the -`Python Packaging User Guide. `_ - ``main`` functions are often used to create command-line tools by specifying them as entry points for console scripts. When this is done, `pip `_ inserts the function call into a template script, @@ -139,6 +138,12 @@ your program will have an exit code of ``1``, indicating failure, and the string will be written to :data:`sys.stderr`. The ``echo.py`` example from earlier exemplifies using the ``sys.exit(main())`` convention. +.. seealso:: + + `Python Packaging User Guide `_ + contains a collection of tutorials and references on how to distribute and + install Python packages with modern tools. + ``__main__.py`` in Python Packages ---------------------------------- @@ -160,10 +165,10 @@ directly from the command line using the :option:`-m` flag. For example:: python3 -m bandclass -This command will cause ``__main__.py`` to run. For more details about the -:option:`-m` flag, see :mod:`runpy`. How you utilize this mechanism will depend -on the nature of the package you are writing, but in this hypothetical case, it -might make sense to allow the teacher to search for students:: +This command will cause ``__main__.py`` to run. How you utilize this mechanism +will depend on the nature of the package you are writing, but in this +hypothetical case, it might make sense to allow the teacher to search for +students:: # bandclass/__main__.py @@ -178,9 +183,6 @@ import. This import style must be used when referencing modules within a package. For more details, see :ref:`intra-package-references` in the :ref:`tut-modules` section of the tutorial. -For an example of a package using ``__main__.py`` in our standard library, see -:mod:`venv`, and its invocation via ``python3 -m venv [directory]``. - Idiomatic Usage ^^^^^^^^^^^^^^^ @@ -197,8 +199,23 @@ attribute will include the package's path if imported:: >>> asyncio.__main__.__name__ 'asyncio.__main__' -That being said, minimal ``__main__.py`` like the :mod:`venv` one mentioned -above are preferred. +This won't work for ``__main__.py`` files in the root directory of a .zip file +though. Hence, for consistency, minimal ``__main__.py`` like the :mod:`venv` +one mentioned above are preferred. + +.. seealso:: + + See :mod:`venv` for an example of a package with a minimal ``__main__.py`` + in the standard library. It doesn't contain a ``if __name__ == '__main__'`` + block. You can invoke it with ``python3 -m venv [directory]``. + + See :mod:`runpy` for more details on the :option:`-m` flag to the + interpreter executable. + + See :mod:`zipapp` for how to run applications packaged as *.zip* files. In + this case Python looks for a ``__main__.py`` file in the root directory of + the archive. + ``import __main__`` From 1e86e026d4b1792bef50763d2aa88832db36bfae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 15:43:39 +0200 Subject: [PATCH 41/46] Appease `make suspicious` --- Doc/tools/susp-ignored.csv | 1 + 1 file changed, 1 insertion(+) diff --git a/Doc/tools/susp-ignored.csv b/Doc/tools/susp-ignored.csv index 1fde253feac2fa..8db0f3bb4bf799 100644 --- a/Doc/tools/susp-ignored.csv +++ b/Doc/tools/susp-ignored.csv @@ -110,6 +110,7 @@ howto/pyporting,,::,Programming Language :: Python :: 3 howto/regex,,::, howto/regex,,:foo,(?:foo) howto/urllib2,,:password,"""joe:password@example.com""" +library/__main__,,`, library/ast,,:upper,lower:upper library/ast,,:step,lower:upper:step library/audioop,,:ipos,"# factor = audioop.findfactor(in_test[ipos*2:ipos*2+len(out_test)]," From 9c8744261c20421530287236cd346dc61bb59c8b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 15:43:56 +0200 Subject: [PATCH 42/46] Spell out examples of top-level code environments --- Doc/library/__main__.rst | 65 ++++++++++++++++++++++++++++++++++------ 1 file changed, 56 insertions(+), 9 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 5dbb3f37fcf7cc..4108821eac9743 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -41,11 +41,56 @@ package's path:: In some circumstances, ``__name__`` is set to the string ``'__main__'``. ``__main__`` is the name of the environment where top-level code is run. -"Top-level code" means when a Python module is initialized from an interactive -prompt, from standard input, from a file argument, with the :option:`-c` -argument, or with the :option:`-m` argument, but **not** when it is initialized -from an import statement. In any of these situations, the module's -``__name__`` is set to ``'__main__'``. +"Top-level code" is the first user-specified Python module that starts running. +It's "top-level" because it imports all other modules that the program needs. +Sometimes "top-level code" is called an *entry point* to the application. + +The top-level code environment can be: + +* the scope of an interactive prompt:: + + >>> __name__ + '__main__' + +* the Python module passed to the Python interpreter as a file argument: + + .. code-block:: shell-session + + $ python3 helloworld.py + Hello, world! + +* the Python module or package passed to the Python interpreter with the + :option:`-m` argument: + + .. code-block:: shell-session + + $ python3 -m tarfile + usage: tarfile.py [-h] [-v] (...) + +* Python code read by the Python interpreter from standard input: + + .. code-block:: shell-session + + $ echo "import this" | python3 + The Zen of Python, by Tim Peters + + Beautiful is better than ugly. + Explicit is better than implicit. + ... + +* Python code passed to the Python interpreter with the :option:`-c` argument: + + .. code-block:: shell-session + + $ python3 -c "import this" + The Zen of Python, by Tim Peters + + Beautiful is better than ugly. + Explicit is better than implicit. + ... + +In each of these situations, the top-level module's ``__name__`` is set to +``'__main__'``. As a result, a module can discover whether or not it is running in the top-level environment by checking its own ``__name__``, which allows a common @@ -161,9 +206,11 @@ package, "bandclass": └── student.py ``__main__.py`` will be executed when the package itself is invoked -directly from the command line using the :option:`-m` flag. For example:: +directly from the command line using the :option:`-m` flag. For example: + +.. code-block:: shell-session - python3 -m bandclass + $ python3 -m bandclass This command will cause ``__main__.py`` to run. How you utilize this mechanism will depend on the nature of the package you are writing, but in this @@ -266,7 +313,7 @@ Example usage of this module could be as follows:: Now, if we started our program, the result would look like this: -.. code-block:: text +.. code-block:: shell-session $ python3 start.py Define the variable `my_name`! @@ -275,7 +322,7 @@ The exit code of the program would be 1, indicating an error. Uncommenting the line with ``my_name = "Dinsdale"`` fixes the program and now it exits with status code 0, indicating success: -.. code-block:: text +.. code-block:: shell-session $ python3 start.py Dinsdale found in file /path/to/start.py From 0d4fc8a31a05a8c9a6ade86290ab74eec340d4ea Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 15:57:50 +0200 Subject: [PATCH 43/46] Appease double dot alignment aesthetics --- Doc/library/__main__.rst | 64 ++++++++++++++++++++-------------------- 1 file changed, 32 insertions(+), 32 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 4108821eac9743..160b77ee9a0edc 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -56,38 +56,38 @@ The top-level code environment can be: .. code-block:: shell-session - $ python3 helloworld.py - Hello, world! + $ python3 helloworld.py + Hello, world! * the Python module or package passed to the Python interpreter with the :option:`-m` argument: .. code-block:: shell-session - $ python3 -m tarfile - usage: tarfile.py [-h] [-v] (...) + $ python3 -m tarfile + usage: tarfile.py [-h] [-v] (...) * Python code read by the Python interpreter from standard input: .. code-block:: shell-session - $ echo "import this" | python3 - The Zen of Python, by Tim Peters + $ echo "import this" | python3 + The Zen of Python, by Tim Peters - Beautiful is better than ugly. - Explicit is better than implicit. - ... + Beautiful is better than ugly. + Explicit is better than implicit. + ... * Python code passed to the Python interpreter with the :option:`-c` argument: .. code-block:: shell-session - $ python3 -c "import this" - The Zen of Python, by Tim Peters + $ python3 -c "import this" + The Zen of Python, by Tim Peters - Beautiful is better than ugly. - Explicit is better than implicit. - ... + Beautiful is better than ugly. + Explicit is better than implicit. + ... In each of these situations, the top-level module's ``__name__`` is set to ``'__main__'``. @@ -103,8 +103,8 @@ an import statement:: .. seealso:: - For a more detailed look at how ``__name__`` is set in all situations, see - the tutorial section :ref:`tut-modules`. + For a more detailed look at how ``__name__`` is set in all situations, see + the tutorial section :ref:`tut-modules`. Idiomatic Usage @@ -185,9 +185,9 @@ earlier exemplifies using the ``sys.exit(main())`` convention. .. seealso:: - `Python Packaging User Guide `_ - contains a collection of tutorials and references on how to distribute and - install Python packages with modern tools. + `Python Packaging User Guide `_ + contains a collection of tutorials and references on how to distribute and + install Python packages with modern tools. ``__main__.py`` in Python Packages @@ -210,7 +210,7 @@ directly from the command line using the :option:`-m` flag. For example: .. code-block:: shell-session - $ python3 -m bandclass + $ python3 -m bandclass This command will cause ``__main__.py`` to run. How you utilize this mechanism will depend on the nature of the package you are writing, but in this @@ -252,16 +252,16 @@ one mentioned above are preferred. .. seealso:: - See :mod:`venv` for an example of a package with a minimal ``__main__.py`` - in the standard library. It doesn't contain a ``if __name__ == '__main__'`` - block. You can invoke it with ``python3 -m venv [directory]``. + See :mod:`venv` for an example of a package with a minimal ``__main__.py`` + in the standard library. It doesn't contain a ``if __name__ == '__main__'`` + block. You can invoke it with ``python3 -m venv [directory]``. - See :mod:`runpy` for more details on the :option:`-m` flag to the - interpreter executable. + See :mod:`runpy` for more details on the :option:`-m` flag to the + interpreter executable. - See :mod:`zipapp` for how to run applications packaged as *.zip* files. In - this case Python looks for a ``__main__.py`` file in the root directory of - the archive. + See :mod:`zipapp` for how to run applications packaged as *.zip* files. In + this case Python looks for a ``__main__.py`` file in the root directory of + the archive. @@ -315,8 +315,8 @@ Now, if we started our program, the result would look like this: .. code-block:: shell-session - $ python3 start.py - Define the variable `my_name`! + $ python3 start.py + Define the variable `my_name`! The exit code of the program would be 1, indicating an error. Uncommenting the line with ``my_name = "Dinsdale"`` fixes the program and now it exits with @@ -324,8 +324,8 @@ status code 0, indicating success: .. code-block:: shell-session - $ python3 start.py - Dinsdale found in file /path/to/start.py + $ python3 start.py + Dinsdale found in file /path/to/start.py Note that importing ``__main__`` doesn't cause any issues with unintentionally running top-level code meant for script use which is put in the From 4e51333abb4c72e16f61b0bb5f3105c562615303 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 16:28:40 +0200 Subject: [PATCH 44/46] Replace lies with truth --- Doc/library/__main__.rst | 17 ++++++++++++----- Doc/reference/import.rst | 2 ++ 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index 160b77ee9a0edc..b31b22e98c6f3d 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -329,11 +329,18 @@ status code 0, indicating success: Note that importing ``__main__`` doesn't cause any issues with unintentionally running top-level code meant for script use which is put in the -``if __name__ == "__main__"`` block of ``start.py``. It's because this code -has already executed at the start of the program and the module is already -fully loaded when ``namely.py`` imports ``__main__``. - -The Python REPL is one example of a "top-level environment", so anything +``if __name__ == "__main__"`` block of ``start.py``. Why does this work? + +Python inserts an empty ``__main__`` module in :attr:`sys.modules` at +interpreter startup, and populates it by running top-level code. In our example +this is the ``start.py`` file which runs line by line and imports ``namely``. +In turn, ``namely.py`` imports ``__main__`` (which is ``start.py``). That's an +import cycle! Fortunately, since the partially populated ``__main__`` +module is present in :attr:`sys.modules`, Python passes that to ``namely.py``. +See :ref:`Special considerations for __main__ ` in the +import system's reference for details on how this works. + +The Python REPL is another example of a "top-level environment", so anything defined in the REPL becomes part of the ``__main__`` scope:: >>> import namely diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst index 81a124f745ab6a..39fcba015b6947 100644 --- a/Doc/reference/import.rst +++ b/Doc/reference/import.rst @@ -975,6 +975,8 @@ should expose ``XXX.YYY.ZZZ`` as a usable expression, but .moduleY is not a valid expression. +.. _import-dunder-main: + Special considerations for __main__ =================================== From 6f0f82cd7aaea30dc295ed3b3ff7c1c708f07b9b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 16:38:00 +0200 Subject: [PATCH 45/46] Improve flow introducing what "top-level code environment" is --- Doc/library/__main__.rst | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index b31b22e98c6f3d..d087fe6b4f1e49 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -39,7 +39,12 @@ package's path:: >>> process.__name__ 'concurrent.futures.process' -In some circumstances, ``__name__`` is set to the string ``'__main__'``. +However, if the module is executed in the top-level code environment, +its ``__name__`` is set to the string ``'__main__'``. + +What is the "top-level code environment"? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + ``__main__`` is the name of the environment where top-level code is run. "Top-level code" is the first user-specified Python module that starts running. It's "top-level" because it imports all other modules that the program needs. From 46e7668187e5478635a6ecd58cf9ffde388351c3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Langa?= Date: Tue, 24 Aug 2021 18:37:45 +0200 Subject: [PATCH 46/46] Use module names consistently in example --- Doc/library/__main__.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/library/__main__.rst b/Doc/library/__main__.rst index d087fe6b4f1e49..116a9a9d1d729f 100644 --- a/Doc/library/__main__.rst +++ b/Doc/library/__main__.rst @@ -334,14 +334,14 @@ status code 0, indicating success: Note that importing ``__main__`` doesn't cause any issues with unintentionally running top-level code meant for script use which is put in the -``if __name__ == "__main__"`` block of ``start.py``. Why does this work? +``if __name__ == "__main__"`` block of the ``start`` module. Why does this work? Python inserts an empty ``__main__`` module in :attr:`sys.modules` at interpreter startup, and populates it by running top-level code. In our example -this is the ``start.py`` file which runs line by line and imports ``namely``. -In turn, ``namely.py`` imports ``__main__`` (which is ``start.py``). That's an +this is the ``start`` module which runs line by line and imports ``namely``. +In turn, ``namely`` imports ``__main__`` (which is really ``start``). That's an import cycle! Fortunately, since the partially populated ``__main__`` -module is present in :attr:`sys.modules`, Python passes that to ``namely.py``. +module is present in :attr:`sys.modules`, Python passes that to ``namely``. See :ref:`Special considerations for __main__ ` in the import system's reference for details on how this works.