sortperf.py is broken

[pochmann3]

Bug report

Bug description:

It's supposed to measure sort speed for different kinds of data, from best to worst case. But currently it shows about the same speed for all kinds, because the same list object is sorted a thousand times. Which means all but the first sort are just sorting the already sorted list. And that dominates the total time, making the first=real sort insignificant.

Oddly it was specifically changed to be this way, here, from

    def _prepare_data(self, loops: int) -> list[float]:
        bench = BENCHMARKS[self._name]
        return [
            bench(self._size, self._random)
            for _ in range(loops)
        ]

to:

    def _prepare_data(self, loops: int) -> list[float]:
        bench = BENCHMARKS[self._name]
        return [bench(self._size, self._random)] * loops

There was even a comment showing that all cases from best to worst now take about the same tiny time 29.2 us, calling it an optimization (and I think it refers to the above change). So is this intentional/desired? I highly doubt that, but if it's a mistake, it's really weird.

The whole update was btw called "Move Lib/test/sortperf.py to Tools/scripts", but in reality it wasn't just moved but completely rewritten. I don't think that was good, in my opinion moves should just be moves, not hide massive changes.

@sobolevn @tim-one

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-116576: Fix Tools/scripts/sortperf.py sorting the same list #116577
• [3.12] gh-116576: Fix Tools/scripts/sortperf.py sorting the same list (GH-116577) #116582
• [3.11] gh-116576: Fix Tools/scripts/sortperf.py sorting the same list (GH-116577) #116583

[sobolevn]

Indeed, thanks for noticing! 👍

[pochmann3]

Thanks, looks alright now, both the code fix and the times in the PR comment.