make test: module test that failed doesn't exist

[pmatos]

BPO	31201
Nosy	@ned-deily, @bitdancer, @sdownum, @pmatos
PRs	bpo-31201: Clarify command to re-run failing test(s) with example #6417 [3.7] bpo-31201: Clarify command to re-run failing test(s) with example (GH-6417) #6437 [3.6] bpo-31201: Clarify command to re-run failing test(s) with example (GH-6417) #6438

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-04-09.21:21:01.713>
created_at = <Date 2017-08-14.10:43:48.151>
labels = ['easy', '3.7', '3.8', 'docs']
title = "make test: module test that failed doesn't exist"
updated_at = <Date 2018-04-09.21:21:01.711>
user = 'https://github.com/pmatos'

bugs.python.org fields:

activity = <Date 2018-04-09.21:21:01.711>
actor = 'ned.deily'
assignee = 'docs@python'
closed = True
closed_date = <Date 2018-04-09.21:21:01.713>
closer = 'ned.deily'
components = ['Documentation']
creation = <Date 2017-08-14.10:43:48.151>
creator = 'pmatos'
dependencies = []
files = []
hgrepos = []
issue_num = 31201
keywords = ['patch', 'easy']
message_count = 8.0
messages = ['300241', '300250', '300251', '300252', '315149', '315150', '315151', '315152']
nosy_count = 5.0
nosy_names = ['ned.deily', 'r.david.murray', 'docs@python', 'steven.downum', 'pmatos']
pr_nums = ['6417', '6437', '6438']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue31201'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[pmatos]

Trying to rerun the tests that failed but it just says that the module doesn't exist. I am running the exact command line mentioned in the readme file:

$ make test TESTOPTS="-v test_that_failed"
running build
running build_ext

The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
atexit pwd time

running build_scripts
copying and adjusting /home/pmatos/Projects/cpython/Tools/scripts/pydoc3 -> build/scripts-3.7
copying and adjusting /home/pmatos/Projects/cpython/Tools/scripts/idle3 -> build/scripts-3.7
copying and adjusting /home/pmatos/Projects/cpython/Tools/scripts/2to3 -> build/scripts-3.7
copying and adjusting /home/pmatos/Projects/cpython/Tools/scripts/pyvenv -> build/scripts-3.7
changing mode of build/scripts-3.7/pydoc3 from 664 to 775
changing mode of build/scripts-3.7/idle3 from 664 to 775
changing mode of build/scripts-3.7/2to3 from 664 to 775
changing mode of build/scripts-3.7/pyvenv from 664 to 775
renaming build/scripts-3.7/pydoc3 to build/scripts-3.7/pydoc3.7
renaming build/scripts-3.7/idle3 to build/scripts-3.7/idle3.7
renaming build/scripts-3.7/2to3 to build/scripts-3.7/2to3-3.7
renaming build/scripts-3.7/pyvenv to build/scripts-3.7/pyvenv-3.7
./python  ../Tools/scripts/run_tests.py -v test_that_failed
/home/pmatos/Projects/cpython/debug/python -u -W default -bb -E -W error::BytesWarning -m test -r -w -j 0 -u all,-largefile,-audio,-gui -v test_that_failed
== CPython 3.7.0a0 (heads/master:48d9823a0e, Aug 14 2017, 11:53:56) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]
== Linux-4.11.11-300.fc26.x86_64-x86_64-with-fedora-26-Twenty_Six little-endian
== hash algorithm: siphash24 64bit
== cwd: /home/pmatos/Projects/cpython/debug/build/test_python_22582
== CPU count: 16
== encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, verbose=0, bytes_warning=2, quiet=0, hash_randomization=1, isolated=0)
Using random seed 8085058
Run tests in parallel using 18 child processes
0:00:00 load avg: 15.90 [1/1/1] test_that_failed failed
test test_that_failed crashed -- Traceback (most recent call last):
  File "/home/pmatos/Projects/cpython/Lib/test/libregrtest/runtest.py", line 163, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/home/pmatos/Projects/cpython/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'test.test_that_failed'

1 test failed:
    test_that_failed
Re-running failed tests in verbose mode
Re-running test 'test_that_failed' in verbose mode
test test_that_failed crashed -- Traceback (most recent call last):
  File "/home/pmatos/Projects/cpython/Lib/test/libregrtest/runtest.py", line 163, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/home/pmatos/Projects/cpython/Lib/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'test.test_that_failed'

1 test failed again:
test_that_failed

Total duration: 355 ms
Tests result: FAILURE
make: *** [Makefile:1019: test] Error 2

[bitdancer]

Replace "test_that_failed" with the name of the test that failed.

The README could be improved by saying:

If any tests fail, you can re-run the failing test(s) in verbose mode. For example if, 'test_os' and 'test_gdb' failed, you can run::

 make test TESTOPTS="-v test_os test_gdb"

[pmatos]

Argh, apologies David. I hadn't noticed the mistake. I simply copy/pasted and was expecting the command line to rerun all the tests that had previously failed. I was assuming the suite had some kind of state that recorded the tests that previously failed and ran only those. :)

[bitdancer]

Yep, I figured that. That's why I suggested the clarification to the README, if someone wants to generate a PR for it.

[ned-deily]

New changeset 0f914b5 by Ned Deily (Aaron Ang) in branch 'master':
bpo-31201: Clarify command to re-run failing test(s) with example (GH-6417)
0f914b5

[ned-deily]

New changeset 71a3837 by Ned Deily (Miss Islington (bot)) in branch '3.7':
[3.7] bpo-31201: Clarify command to re-run failing test(s) with example (GH-6417) (GH-6437)
71a3837

[ned-deily]

New changeset 83bb399 by Ned Deily (Miss Islington (bot)) in branch '3.6':
[3.6] bpo-31201: Clarify command to re-run failing test(s) with example (GH-6417) (GH-6438)
83bb399

[ned-deily]

Thanks for the PR, Aaron!