Skip to content

gh-117364: Add doctest.SkipTest to skip all or some doctest examples #122935

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions Doc/library/doctest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -524,6 +524,41 @@ Some details you should read once, but won't need to remember:
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'


Skipping Tests
^^^^^^^^^^^^^^

You can skip all or some test examples with :exc:`SkipTest` exception.

Let's say that you have some Python-version-specific API:

.. doctest::

>>> import sys

>>> if sys.version_info >= (3, 14):
... def process_data():
... return 'processed data'

And you want to mention it in the doctest and only run it
for the appropriate Python versions:

.. doctest::

>>> if sys.version_info < (3, 14):
... import doctest
... raise doctest.SkipTest('This test is only for 3.14+')

>>> # This line and below will only be executed by doctest on 3.14+
>>> process_data()
'processed data'

.. exception:: SkipTest

Special exception after raising it, all test examples will be skipped.

.. versionadded:: 3.14


.. _option-flags-and-directives:
.. _doctest-options:

Expand Down Expand Up @@ -631,6 +666,8 @@ doctest decides whether actual output matches an example's expected output:

The SKIP flag can also be used for temporarily "commenting out" examples.

See also: :exc:`~SkipTest`.


.. data:: COMPARISON_FLAGS

Expand Down
29 changes: 28 additions & 1 deletion Lib/doctest.py
Original file line number Diff line number Diff line change
Expand Up @@ -1354,6 +1354,7 @@ def __run(self, test, compileflags, out):
SUCCESS, FAILURE, BOOM = range(3) # `outcome` state

check = self._checker.check_output
skip_following_examples = False

# Process each example.
for examplenum, example in enumerate(test.examples):
Expand All @@ -1374,7 +1375,8 @@ def __run(self, test, compileflags, out):
self.optionflags &= ~optionflag

# If 'SKIP' is set, then skip this example.
if self.optionflags & SKIP:
# Or if `SkipTest` was raised, skip all following examples.
if skip_following_examples or self.optionflags & SKIP:
skips += 1
continue

Expand All @@ -1400,10 +1402,16 @@ def __run(self, test, compileflags, out):
raise
except:
exception = sys.exc_info()
exc = exception[0]
if f'{exc.__module__}.{exc.__qualname__}' == 'doctest.SkipTest':
skip_following_examples = True
self.debugger.set_continue() # ==== Example Finished ====

got = self._fakeout.getvalue() # the actual output
self._fakeout.truncate(0)
if skip_following_examples:
skips += 1
continue
outcome = FAILURE # guilty until proved innocent or insane

# If the example executed without raising any exceptions,
Expand Down Expand Up @@ -2230,6 +2238,25 @@ def run_docstring_examples(f, globs, verbose=False, name="NoName",
for test in finder.find(f, name, globs=globs):
runner.run(test, compileflags=compileflags)


class SkipTest(Exception):
"""
Special exception that will skip all following examples.

Can be used to conditionally skip some doctests or its parts:

>>> import doctest
>>> 1 + 1 # will be checked
2

>>> raise doctest.SkipTest("Do not check any example after this line")

>>> 1 + 1 # won't be checked!
3

"""


######################################################################
## 7. Unittest Support
######################################################################
Expand Down
25 changes: 21 additions & 4 deletions Lib/test/test_doctest/test_doctest.py
Original file line number Diff line number Diff line change
Expand Up @@ -1950,6 +1950,22 @@ def option_directives(): r"""
ValueError: line 0 of the doctest for s has an option directive on a line with no example: '# doctest: +ELLIPSIS'

>>> _colorize.COLORIZE = save_colorize

You can skip following examples via `raise SkipTest`:

>>> def f(x):
... r'''
... >>> print(1) # first success
... 1
... >>> raise doctest.SkipTest("All tests after this line will be skipped")
... >>> print(4) # second skipped success
... 4
... >>> print(5) # first skipped failure
... 500
... '''
>>> test = doctest.DocTestFinder().find(f)[0]
>>> doctest.DocTestRunner(verbose=False).run(test)
TestResults(failed=0, attempted=4, skipped=3)
"""

def test_testsource(): r"""
Expand Down Expand Up @@ -2474,12 +2490,13 @@ def test_DocFileSuite():

>>> suite = doctest.DocFileSuite('test_doctest.txt',
... 'test_doctest4.txt',
... 'test_doctest_skip.txt')
... 'test_doctest_skip.txt',
... 'test_doctest_skip2.txt')
>>> result = suite.run(unittest.TestResult())
>>> result
<unittest.result.TestResult run=3 errors=0 failures=1>
>>> len(result.skipped)
1
<unittest.result.TestResult run=4 errors=0 failures=1>
>>> len(result.skipped) # not all examples in test_doctest_skip2 are skipped
1

You can specify initial global variables:

Expand Down
6 changes: 6 additions & 0 deletions Lib/test/test_doctest/test_doctest_skip2.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
This is a sample doctest in a text file, in which all examples are skipped.

>>> import doctest # this example is not skipped
>>> raise doctest.SkipTest("All tests after this line are skipped")
>>> 2 + 2
5
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add :exc:`doctest.SkipTest` to conditionally skip all or some doctest
examples.
Loading