Nondeterministic type checking on pytorch/vision

[hauntsaninja]

Repro with e.g. mypy_primer --new 'v1.15' --old 'v1.15' -k vision --debug:

diff --git a/mypy_primer/projects.py b/mypy_primer/projects.py
index 09c050c..0309cff 100644
--- a/mypy_primer/projects.py
+++ b/mypy_primer/projects.py
@@ -547,7 +547,7 @@ def get_projects() -> list[Project]:
         ),
         Project(
             location="https://github.com/pytorch/vision",
-            mypy_cmd=None,  # "{mypy}",
+            mypy_cmd="{mypy}",
             pyright_cmd="{pyright}",
             deps=["numpy", "pillow"],

Linking hauntsaninja/mypy_primer#161

[sterliakov]

One more example with random primer diffs on xarray, it only started appearing recently: #19102 (comment)

[hauntsaninja]

xarray might be a newer regression, since I can't repro with a few runs of mypy_primer --new 'v1.15' --old 'v1.15' -k pydata/xarray --debug --output concise. I'll try to bisect it later

[sterliakov]

Almost sure that xarray will bisect to #18976 (where it appeared for the first time and was expected), but that's not the cause - xarray indeterminism looks like the problem with noncommutative meet/join we are already aware of. I wasn't able to repro it - any MRE I managed to think of fails every time, not randomly.

…

On Fri, May 23, 2025, 22:50 Shantanu ***@***.***> wrote: *hauntsaninja* left a comment (python/mypy#19121) <#19121 (comment)> xarray might be a newer regression, since I can't repro with a few runs of mypy_primer --new 'v1.15' --old 'v1.15' -k pydata/xarray --debug --output concise. I'll try to bisect it later — Reply to this email directly, view it on GitHub <#19121 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMBQIRDYJP63OYNQYXC24KL276CX5AVCNFSM6AAAAAB5RKR64SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMBVG44DCMBZHA> . You are receiving this because you commented.Message ID: ***@***.***>

[hauntsaninja]

Simple minimised version of torchvision:

printf '
def draw(
    colors1: list[str | tuple[int, int, int]] | str | tuple[int, int, int] | None,
    colors2: list[str | tuple[int, int, int]] | str | tuple[int, int, int] | None,
) -> None:
    for c1, c2 in zip(colors1, colors2):
        reveal_type(c1)
        reveal_type(c2)
' > test.py


first_output="$(mypy test.py)"
echo "$first_output"
echo "----------"
for run in {2..10}; do
  current_output="$(mypy test.py)"
  echo "$current_output"
  echo "----------"
  if [[ "$current_output" != "$first_output" ]]; then
    echo output mismatch
    exit 1
  fi
done

Also linking #16979 and #18125