Merge pull request #3931 from nicoddemus/internal-warnings

Use standard warnings for internal pytest warnings

Author: nicoddemus

changelog/2452.feature.rst

@@ -0,0 +1,5 @@
+Internal pytest warnings are now issued using the standard ``warnings`` module, making it possible to use
+the standard warnings filters to manage those warnings. This introduces ``PytestWarning``,
+``PytestDeprecationWarning`` and ``RemovedInPytest4Warning`` warning types as part of the public API.
+
+Consult `the documentation <https://docs.pytest.org/en/latest/warnings.html#internal-pytest-warnings>`_ for more info.

changelog/2452.removal.rst

@@ -0,0 +1,12 @@
+``Config.warn`` has been deprecated, it should be replaced by calls to the standard ``warnings.warn``.
+
+``Node.warn`` now supports two signatures:
+
+* ``node.warn(PytestWarning("some message"))``: is now the recommended way to call this function. The warning
+  instance must be a ``PytestWarning`` or subclass  instance.
+
+* ``node.warn("CI", "some message")``: this code/message form is now deprecated and should be converted to
+  the warning instance form above.
+
+``RemovedInPytest4Warning`` and ``PytestExperimentalApiWarning`` are now part of the public API and should be accessed
+using ``pytest.RemovedInPytest4Warning`` and ``pytest.PytestExperimentalApiWarning``.

changelog/2908.feature.rst

@@ -0,0 +1,5 @@
+``DeprecationWarning`` and ``PendingDeprecationWarning`` are now shown by default if no other warning filter is
+configured. This makes pytest more compliant with
+`PEP-0506 <https://www.python.org/dev/peps/pep-0565/#recommended-filter-settings-for-test-runners>`_. See
+`the docs <https://docs.pytest.org/en/latest/warnings.html#deprecationwarning-and-pendingdeprecationwarning>`_ for
+more info.

changelog/3251.feture.rst

@@ -0,0 +1 @@
+Warnings are now captured and displayed during test collection.

changelog/3936.removal.rst

@@ -0,0 +1,5 @@
+``@pytest.mark.filterwarnings`` second parameter is no longer regex-escaped,
+making it possible to actually use regular expressions to check the warning message.
+
+**Note**: regex-escaping the match string was an implementation oversight that might break test suites which depend
+on the old behavior.

doc/en/reference.rst

@@ -611,6 +611,8 @@ Session related reporting hooks:
 .. autofunction:: pytest_terminal_summary
 .. autofunction:: pytest_fixture_setup
 .. autofunction:: pytest_fixture_post_finalizer
+.. autofunction:: pytest_logwarning
+.. autofunction:: pytest_warning_captured
 
 And here is the central hook for reporting about
 test execution:

doc/en/warnings.rst

@@ -36,8 +36,6 @@ Running pytest now produces this output::
     -- Docs: https://docs.pytest.org/en/latest/warnings.html
     =================== 1 passed, 1 warnings in 0.12 seconds ===================
 
-Pytest by default catches all warnings except for ``DeprecationWarning`` and ``PendingDeprecationWarning``.
-
 The ``-W`` flag can be passed to control which warnings will be displayed or even turn
 them into errors::
 
@@ -78,6 +76,53 @@ Both ``-W`` command-line option and ``filterwarnings`` ini option are based on P
 `-W option`_ and `warnings.simplefilter`_, so please refer to those sections in the Python
 documentation for other examples and advanced usage.
 
+Disabling warning summary
+-------------------------
+
+Although not recommended, you can use the ``--disable-warnings`` command-line option to suppress the
+warning summary entirely from the test run output.
+
+Disabling warning capture entirely
+----------------------------------
+
+This plugin is enabled by default but can be disabled entirely in your ``pytest.ini`` file with:
+
+    .. code-block:: ini
+
+        [pytest]
+        addopts = -p no:warnings
+
+Or passing ``-p no:warnings`` in the command-line. This might be useful if your test suites handles warnings
+using an external system.
+
+
+.. _`deprecation-warnings`:
+
+DeprecationWarning and PendingDeprecationWarning
+------------------------------------------------
+
+.. versionadded:: 3.8
+
+By default pytest will display ``DeprecationWarning`` and ``PendingDeprecationWarning`` if no other warning filters
+are configured.
+
+To disable showing ``DeprecationWarning`` and ``PendingDeprecationWarning`` warnings, you might define any warnings
+filter either in the command-line or in the ini file, or you can use:
+
+.. code-block:: ini
+
+    [pytest]
+    filterwarnings =
+        ignore::DeprecationWarning
+        ignore::PendingDeprecationWarning
+
+.. note::
+    This makes pytest more compliant with `PEP-0506 <https://www.python.org/dev/peps/pep-0565/#recommended-filter-settings-for-test-runners>`_ which suggests that those warnings should
+    be shown by default by test runners, but pytest doesn't follow ``PEP-0506`` completely because resetting all
+    warning filters like suggested in the PEP will break existing test suites that configure warning filters themselves
+    by calling ``warnings.simplefilter`` (see issue `#2430 <https://github.com/pytest-dev/pytest/issues/2430>`_
+    for an example of that).
+
 
 .. _`filterwarnings`:
 
@@ -144,18 +189,6 @@ decorator or to all tests in a module by setting the ``pytestmark`` variable:
 .. _`pytest-warnings`: https://github.com/fschulze/pytest-warnings
 
 
-Disabling warning capture
--------------------------
-
-This feature is enabled by default but can be disabled entirely in your ``pytest.ini`` file with:
-
-    .. code-block:: ini
-
-        [pytest]
-        addopts = -p no:warnings
-
-Or passing ``-p no:warnings`` in the command-line.
-
 .. _`asserting warnings`:
 
 .. _assertwarnings:
@@ -296,3 +329,53 @@ You can also use it as a contextmanager::
     def test_global():
         with pytest.deprecated_call():
             myobject.deprecated_method()
+
+
+Internal pytest warnings
+------------------------
+
+.. versionadded:: 3.8
+
+pytest may generate its own warnings in some situations, such as improper usage or deprecated features.
+
+For example, pytest will emit a warning if it encounters a class that matches :confval:`python_classes` but also
+defines an ``__init__`` constructor, as this prevents the class from being instantiated:
+
+.. code-block:: python
+
+    # content of test_pytest_warnings.py
+    class Test:
+        def __init__(self):
+            pass
+
+        def test_foo(self):
+            assert 1 == 1
+
+::
+
+    $ pytest test_pytest_warnings.py -q
+    ======================================== warnings summary =========================================
+    test_pytest_warnings.py:1
+      $REGENDOC_TMPDIR/test_pytest_warnings.py:1: PytestWarning: cannot collect test class 'Test' because it has a __init__ constructor
+        class Test:
+
+    -- Docs: http://doc.pytest.org/en/latest/warnings.html
+    1 warnings in 0.01 seconds
+
+
+
+These warnings might be filtered using the same builtin mechanisms used to filter other types of warnings.
+
+Following our :ref:`backwards-compatibility`, deprecated features will be kept *at least* two minor releases. After that,
+they will changed so they by default raise errors instead of just warnings, so users can adapt to it on their own time
+if not having done so until now. In a later release the deprecated feature will be removed completely.
+
+The following warning types ares used by pytest and are part of the public API:
+
+.. autoclass:: pytest.PytestWarning
+
+.. autoclass:: pytest.PytestDeprecationWarning
+
+.. autoclass:: pytest.RemovedInPytest4Warning
+
+.. autoclass:: pytest.PytestExperimentalApiWarning

doc/en/writing_plugins.rst

@@ -419,7 +419,7 @@ additionally it is possible to copy examples for a example folder before running
 
     ============================= warnings summary =============================
     test_example.py::test_plugin
-      $REGENDOC_TMPDIR/test_example.py:4: PytestExerimentalApiWarning: testdir.copy_example is an experimental api that may change over time
+      $REGENDOC_TMPDIR/test_example.py:4: PytestExperimentalApiWarning: testdir.copy_example is an experimental api that may change over time
         testdir.copy_example("test_example.py")
 
     -- Docs: https://docs.pytest.org/en/latest/warnings.html

src/_pytest/assertion/rewrite.py

@@ -209,8 +209,12 @@ def mark_rewrite(self, *names):
         self._must_rewrite.update(names)
 
     def _warn_already_imported(self, name):
-        self.config.warn(
-            "P1", "Module already imported so cannot be rewritten: %s" % name
+        from _pytest.warning_types import PytestWarning
+        from _pytest.warnings import _issue_config_warning
+
+        _issue_config_warning(
+            PytestWarning("Module already imported so cannot be rewritten: %s" % name),
+            self.config,
         )
 
     def load_module(self, name):
@@ -746,13 +750,17 @@ def visit_Assert(self, assert_):
         the expression is false.
 
         """
-        if isinstance(assert_.test, ast.Tuple) and self.config is not None:
-            fslocation = (self.module_path, assert_.lineno)
-            self.config.warn(
-                "R1",
-                "assertion is always true, perhaps " "remove parentheses?",
-                fslocation=fslocation,
+        if isinstance(assert_.test, ast.Tuple) and len(assert_.test.elts) >= 1:
+            from _pytest.warning_types import PytestWarning
+            import warnings
+
+            warnings.warn_explicit(
+                PytestWarning("assertion is always true, perhaps remove parentheses?"),
+                category=None,
+                filename=str(self.module_path),
+                lineno=assert_.lineno,
             )
+
         self.statements = []
         self.variables = []
         self.variable_counter = itertools.count()

src/_pytest/cacheprovider.py

@@ -33,22 +33,27 @@
 @attr.s
 class Cache(object):
     _cachedir = attr.ib(repr=False)
-    _warn = attr.ib(repr=False)
+    _config = attr.ib(repr=False)
 
     @classmethod
     def for_config(cls, config):
         cachedir = cls.cache_dir_from_config(config)
         if config.getoption("cacheclear") and cachedir.exists():
             shutil.rmtree(str(cachedir))
             cachedir.mkdir()
-        return cls(cachedir, config.warn)
+        return cls(cachedir, config)
 
     @staticmethod
     def cache_dir_from_config(config):
         return paths.resolve_from_str(config.getini("cache_dir"), config.rootdir)
 
     def warn(self, fmt, **args):
-        self._warn(code="I9", message=fmt.format(**args) if args else fmt)
+        from _pytest.warnings import _issue_config_warning
+        from _pytest.warning_types import PytestWarning
+
+        _issue_config_warning(
+            PytestWarning(fmt.format(**args) if args else fmt), self._config
+        )
 
     def makedir(self, name):
         """ return a directory path object with the given name.  If the

src/_pytest/config/__init__.py

@@ -176,7 +176,9 @@ def _prepareconfig(args=None, plugins=None):
                 else:
                     pluginmanager.register(plugin)
         if warning:
-            config.warn("C1", warning)
+            from _pytest.warnings import _issue_config_warning
+
+            _issue_config_warning(warning, config=config)
         return pluginmanager.hook.pytest_cmdline_parse(
             pluginmanager=pluginmanager, args=args
         )
@@ -417,7 +419,12 @@ def _importconftest(self, conftestpath):
                         PYTEST_PLUGINS_FROM_NON_TOP_LEVEL_CONFTEST
                     )
 
-                    warnings.warn(PYTEST_PLUGINS_FROM_NON_TOP_LEVEL_CONFTEST)
+                    warnings.warn_explicit(
+                        PYTEST_PLUGINS_FROM_NON_TOP_LEVEL_CONFTEST,
+                        category=None,
+                        filename=str(conftestpath),
+                        lineno=0,
+                    )
             except Exception:
                 raise ConftestImportFailure(conftestpath, sys.exc_info())
 
@@ -602,7 +609,29 @@ def _ensure_unconfigure(self):
             fin()
 
     def warn(self, code, message, fslocation=None, nodeid=None):
-        """ generate a warning for this test session. """
+        """
+        .. deprecated:: 3.8
+
+            Use :py:func:`warnings.warn` or :py:func:`warnings.warn_explicit` directly instead.
+
+        Generate a warning for this test session.
+        """
+        from _pytest.warning_types import RemovedInPytest4Warning
+
+        if isinstance(fslocation, (tuple, list)) and len(fslocation) > 2:
+            filename, lineno = fslocation[:2]
+        else:
+            filename = "unknown file"
+            lineno = 0
+        msg = "config.warn has been deprecated, use warnings.warn instead"
+        if nodeid:
+            msg = "{}: {}".format(nodeid, msg)
+        warnings.warn_explicit(
+            RemovedInPytest4Warning(msg),
+            category=None,
+            filename=filename,
+            lineno=lineno,
+        )
         self.hook.pytest_logwarning.call_historic(
             kwargs=dict(
                 code=code, message=message, fslocation=fslocation, nodeid=nodeid
@@ -667,8 +696,8 @@ def _initini(self, args):
         r = determine_setup(
             ns.inifilename,
             ns.file_or_dir + unknown_args,
-            warnfunc=self.warn,
             rootdir_cmd_arg=ns.rootdir or None,
+            config=self,
         )
         self.rootdir, self.inifile, self.inicfg = r
         self._parser.extra_info["rootdir"] = self.rootdir