Skip to content

gh-132678: Add --prioritize to regrtest #132679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 18, 2025
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 27 additions & 5 deletions Lib/test/libregrtest/cmdline.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,19 @@
doing memory analysis on the Python interpreter, which process tends to
consume too many resources to run the full regression test non-stop.

-S is used to continue running tests after an aborted run. It will
maintain the order a standard run (ie, this assumes -r is not used).
-S is used to resume running tests after an interrupted run. It will
maintain the order a standard run (i.e. it assumes -r is not used).
This is useful after the tests have prematurely stopped for some external
reason and you want to start running from where you left off rather
than starting from the beginning.
reason and you want to resume the run from where you left off rather
than starting from the beginning. Note: this is different from --prioritize.

--prioritize is used to influence the order of selected tests, such that
the tests listed as an argument are executed first. This is especially
useful when combined with -j and -r to pin the longest-running tests
to start at the beginning of a test run. Pass --prioritize=test_a,test_b
to make test_a run first, followed by test_b, and then the other tests.
If test_a wasn't selected for execution by regular means, --prioritize will
not make it execute.

-f reads the names of tests from the file given as f's argument, one
or more test names per line. Whitespace is ignored. Blank lines and
Expand Down Expand Up @@ -236,7 +244,7 @@ def _create_parser():
help='wait for user input, e.g., allow a debugger '
'to be attached')
group.add_argument('-S', '--start', metavar='START',
help='the name of the test at which to start.' +
help='resume an interrupted run at the following test.' +
more_details)
group.add_argument('-p', '--python', metavar='PYTHON',
help='Command to run Python test subprocesses with.')
Expand All @@ -263,6 +271,10 @@ def _create_parser():
group = parser.add_argument_group('Selecting tests')
group.add_argument('-r', '--randomize', action='store_true',
help='randomize test execution order.' + more_details)
group.add_argument('--prioritize', metavar='TEST1,TEST2,...',
action='append', type=priority_list,
help='select these tests first, even if the order is'
' randomized.' + more_details)
group.add_argument('-f', '--fromfile', metavar='FILE',
help='read names of tests to run from a file.' +
more_details)
Expand Down Expand Up @@ -406,6 +418,10 @@ def resources_list(string):
return u


def priority_list(string):
return [t for t in string.split(",")]


def _parse_args(args, **kwargs):
# Defaults
ns = Namespace()
Expand Down Expand Up @@ -551,4 +567,10 @@ def _parse_args(args, **kwargs):
print(msg, file=sys.stderr, flush=True)
sys.exit(2)

ns.prioritize = [
test
for priority_list in (ns.prioritize or ())
for test in priority_list
]

return ns
11 changes: 11 additions & 0 deletions Lib/test/libregrtest/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ def __init__(self, ns: Namespace, _add_python_opts: bool = False):
self.random_seed = random.getrandbits(32)
else:
self.random_seed = ns.random_seed
self.prioritize_tests: tuple[str, ...] = tuple(ns.prioritize)

self.parallel_threads = ns.parallel_threads

Expand Down Expand Up @@ -237,6 +238,16 @@ def find_tests(self, tests: TestList | None = None) -> tuple[TestTuple, TestList
if self.randomize:
random.shuffle(selected)

for priority_test in reversed(self.prioritize_tests):
try:
selected.remove(priority_test)
except ValueError:
print(f"warning: --prioritize={priority_test} used"
f" but test not actually selected")
continue
else:
selected.insert(0, priority_test)

return (tuple(selected), tests)

@staticmethod
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Add ``--prioritize`` to ``-m test``. This option allows the user to specify
which selected tests should execute first, even if the order is otherwise
randomized. This is particularly useful for tests that run the longest.
Loading