regrtest: simplify regex to match test names for the --fromfile option

[vstinner]

BPO	29035
Nosy	@vstinner, @methane, @serhiy-storchaka, @zhangyangyu
Files	regrtest_regex.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-01-03.00:44:11.556>
created_at = <Date 2016-12-21.14:28:14.137>
labels = ['3.7', 'tests']
title = 'regrtest: simplify regex to match test names for the --fromfile option'
updated_at = <Date 2017-01-03.08:34:58.563>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2017-01-03.08:34:58.563>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-01-03.00:44:11.556>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2016-12-21.14:28:14.137>
creator = 'vstinner'
dependencies = []
files = ['45984']
hgrepos = []
issue_num = 29035
keywords = ['patch']
message_count = 8.0
messages = ['283755', '283756', '283761', '283769', '283774', '284514', '284515', '284544']
nosy_count = 5.0
nosy_names = ['vstinner', 'methane', 'python-dev', 'serhiy.storchaka', 'xiang.zhang']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue29035'
versions = ['Python 3.7']

[vstinner]

Lib/test/libregrtest/main.py uses a complex regex to find "test_builtin" in lines like '0:00:00 [ 4/400] test_builtin -- test_dict took 1 sec'. Recently, I changed (change d8222c197831) the regex to support a filename containing a list of filenames. Example:

haypo@selma$ ls Lib/test/test_*xml*py >| list

haypo@selma$ cat list
Lib/test/test_docxmlrpc.py
Lib/test/test_xml_dom_minicompat.py
Lib/test/test_xml_etree_c.py
Lib/test/test_xml_etree.py
Lib/test/test_xmlrpc_net.py
Lib/test/test_xmlrpc.py

haypo@selma$ ./python -m test --fromfile=list --list
test_docxmlrpc
test_xml_dom_minicompat
test_xml_etree_c
test_xml_etree
test_xmlrpc_net
test_xmlrpc

Serhiy sent me a private message to suggest to simply the regex. So here is a patch.

[vstinner]

FYI initially my idea was to use a very strict to avoid false positives. But when I used the feature, I found that the regex is more annoying than helping (ex: I had to modify the regex to accept filenames with directories). So I now prefer to use a very simple regex matching "test_xxx" anywhere, to have something simple and convenient.

[serhiy-storchaka]

Reading file names from a file looks misleading.

$ ls Lib/ctypes/test/*.py > list
$ head list
Lib/ctypes/test/__init__.py
Lib/ctypes/test/__main__.py
Lib/ctypes/test/test_anon.py
Lib/ctypes/test/test_array_in_pointer.py
Lib/ctypes/test/test_arrays.py
Lib/ctypes/test/test_as_parameter.py
Lib/ctypes/test/test_bitfields.py
Lib/ctypes/test/test_buffers.py
Lib/ctypes/test/test_bytes.py
Lib/ctypes/test/test_byteswap.py
$ ./python -m test --fromfile=list 
Run tests sequentially
0:00:00 [ 1/51] test_anon
test test_anon crashed -- Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/libregrtest/runtest.py", line 152, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/home/serhiy/py/cpython/Lib/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 978, in _gcd_import
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 948, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'test.test_anon'
...

I wouldn't add this feature unless support full file names.

[vstinner]

Ah right, "Lib/ctypes/test/test_anon.py" doesn't work. I don't think
that it's worth to support running directly such test. I don't think
that it's worth it to modify the regex to exclude this case.

--fromfile is written for developers who understand what they do, it's
just an helper. It doesn't prevent any kind of mistakes.

But please keep support for simple lists like "ls
Lib/test/test_*xml*py >| list", it's useful for me.

[serhiy-storchaka]

--fromfile is written for developers who understand what they do, it's
just an helper.

I don't understand the use case of it ;) , but when you need it, the patch LGTM.

You can call just .group() instead of .group(0).

[python-dev]

New changeset a9fe5bee892b by Victor Stinner in branch 'default':
Issue bpo-29035: Simplify a regex in libregrtest
https://hg.python.org/cpython/rev/a9fe5bee892b

[vstinner]

I don't understand the use case of it ;)

When I modify a codec or something related to text codecs, I would like to run all codec tests, not the fully Python test suite. So I use "ls Lib/test/test_*codec*py", the list is quite list, it's annoying to have to copy/paste test names manually.

You can call just .group() instead of .group(0).

Done.

Thanks for your review. I added a new unit test for filenames.

[serhiy-storchaka]

So I use "ls Lib/test/test_*codec*py", the list is quite list, it's annoying to have to copy/paste test names manually.

I would use

./python -m test --list | grep codec | xargs ./python -m test