Skip to content

Recursion error with nested Union #9657

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Peilonrayz opened this issue Oct 28, 2020 · 3 comments · Fixed by #9663
Closed

Recursion error with nested Union #9657

Peilonrayz opened this issue Oct 28, 2020 · 3 comments · Fixed by #9663
Labels

Comments

@Peilonrayz
Copy link
Contributor

Peilonrayz commented Oct 28, 2020

Crash Report

Many of my functions take Iter = Union[Iterator[T], Iterable[T]] as arguments. As for and many other functions in Python work with both iterators and iterables. However when I use Iter[Iter[T]] the code can break.

I've used Iter[Iter[Iter[int]]] as an example as Iter[Iter[T]] didn't break locally on the MVCE. Possibly due to a smaller existing stack.

from typing import Union, Iterator, Iterable


def bar(
    values: Union[
        Iterator[Union[
            Iterator[Union[Iterator[int], Iterable[int]]],
            Iterable[Union[Iterator[int], Iterable[int]]],
        ]],
        Iterable[Union[
            Iterator[Union[Iterator[int], Iterable[int]]],
            Iterable[Union[Iterator[int], Iterable[int]]],
        ]],
    ]
) -> Iterator[int]:
    for i in values:
        for j in i:
            for k in j:
                yield k


bar([[[1]]])

Traceback

$ mkdir example
$ cd example
$ vim test.py
$ mypy --show-traceback test.py >> traceback.txt
test.py:18: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.rtfd.io/en/latest/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.790
test.py:18: : note: use --pdb to drop into pdb
Traceback - it's long
$ cat traceback.txt
Traceback (most recent call last):
  File "/usr/bin/mypy", line 33, in <module>
    sys.exit(load_entry_point('mypy==0.790', 'console_scripts', 'mypy')())
  File "/usr/lib/python3.8/site-packages/mypy/__main__.py", line 8, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/usr/lib/python3.8/site-packages/mypy/main.py", line 90, in main
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/usr/lib/python3.8/site-packages/mypy/build.py", line 180, in build
    result = _build(
  File "/usr/lib/python3.8/site-packages/mypy/build.py", line 254, in _build
    graph = dispatch(sources, manager, stdout)
  File "/usr/lib/python3.8/site-packages/mypy/build.py", line 2630, in dispatch
    process_graph(graph, manager)
  File "/usr/lib/python3.8/site-packages/mypy/build.py", line 2953, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/usr/lib/python3.8/site-packages/mypy/build.py", line 3051, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/usr/lib/python3.8/site-packages/mypy/build.py", line 2111, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 294, in check_first_pass
    self.accept(d)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/usr/lib/python3.8/site-packages/mypy/nodes.py", line 677, in accept
    return visitor.visit_func_def(self)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 726, in visit_func_def
    self._visit_func_def(defn)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 730, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 792, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 975, in check_func_def
    self.accept(item.body)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/usr/lib/python3.8/site-packages/mypy/nodes.py", line 1005, in accept
    return visitor.visit_block(self)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 1973, in visit_block
    self.accept(s)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 401, in accept
    stmt.accept(self)
  File "/usr/lib/python3.8/site-packages/mypy/nodes.py", line 1130, in accept
    return visitor.visit_for_stmt(self)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 3479, in visit_for_stmt
    iterator_type, item_type = self.analyze_iterable_item_type(s.expr)
  File "/usr/lib/python3.8/site-packages/mypy/checker.py", line 3499, in analyze_iterable_item_type
    iterator = echk.check_method_call_by_name('__iter__', iterable, [], [], expr)[0]
  File "/usr/lib/python3.8/site-packages/mypy/checkexpr.py", line 2341, in check_method_call_by_name
    return self.check_union_method_call_by_name(method, base_type,
  File "/usr/lib/python3.8/site-packages/mypy/checkexpr.py", line 2373, in check_union_method_call_by_name
    item, meth_item = self.check_method_call_by_name(method, typ, args, arg_kinds,
  File "/usr/lib/python3.8/site-packages/mypy/checkexpr.py", line 2345, in check_method_call_by_name
    method_type = analyze_member_access(method, base_type, context, False, False, True,
  File "/usr/lib/python3.8/site-packages/mypy/checkmember.py", line 126, in analyze_member_access
    result = _analyze_member_access(name, typ, mx, override_info)
  File "/usr/lib/python3.8/site-packages/mypy/checkmember.py", line 143, in _analyze_member_access
    return analyze_instance_member_access(name, typ, mx, override_info)
  File "/usr/lib/python3.8/site-packages/mypy/checkmember.py", line 225, in analyze_instance_member_access
    return analyze_member_var_access(name, typ, info, mx)
  File "/usr/lib/python3.8/site-packages/mypy/checkmember.py", line 376, in analyze_member_var_access
    return analyze_var(name, v, itype, info, mx, implicit=implicit)
  File "/usr/lib/python3.8/site-packages/mypy/checkmember.py", line 578, in analyze_var
    dispatched_type = meet.meet_types(mx.original_type, itype)
  File "/usr/lib/python3.8/site-packages/mypy/meet.py", line 50, in meet_types
    return t.accept(TypeMeetVisitor(s))
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 1724, in accept
    return visitor.visit_union_type(self)
  File "/usr/lib/python3.8/site-packages/mypy/meet.py", line 449, in visit_union_type
    meets = [meet_types(x, self.s)
  File "/usr/lib/python3.8/site-packages/mypy/meet.py", line 449, in <listcomp>
    meets = [meet_types(x, self.s)
  File "/usr/lib/python3.8/site-packages/mypy/meet.py", line 50, in meet_types
    return t.accept(TypeMeetVisitor(s))
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/meet.py", line 505, in visit_instance
    if is_subtype(t, self.s):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 93, in is_subtype
    return _is_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 135, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 264, in visit_instance
    if right.type.is_protocol and is_protocol_implementation(left, right):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 554, in is_protocol_implementation
    is_compat = is_subtype(subtype, supertype, ignore_pos_arg_names=ignore_names)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 93, in is_subtype
    return _is_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 135, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 1098, in accept
    return visitor.visit_callable_type(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 299, in visit_callable_type
    return is_callable_compatible(
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 857, in is_callable_compatible
    if not ignore_return and not is_compat_return(left.ret_type, right.ret_type):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 193, in _is_subtype
    return is_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 93, in is_subtype
    return _is_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 135, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 258, in visit_instance
    nominal = all(self.check_type_parameter(lefta, righta, tvar.variance)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 258, in <genexpr>
    nominal = all(self.check_type_parameter(lefta, righta, tvar.variance)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 38, in check_type_parameter
    return is_subtype(lefta, righta)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 93, in is_subtype
    return _is_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 135, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 1724, in accept
    return visitor.visit_union_type(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 464, in visit_union_type
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 464, in <genexpr>
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 193, in _is_subtype
    return is_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 93, in is_subtype
    return _is_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 118, in _is_subtype
    is_subtype_of_item = any(is_subtype(orig_left, item,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 118, in <genexpr>
    is_subtype_of_item = any(is_subtype(orig_left, item,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 93, in is_subtype
    return _is_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 135, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 264, in visit_instance
    if right.type.is_protocol and is_protocol_implementation(left, right):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 527, in is_protocol_implementation
    if (mypy.sametypes.is_same_type(l, left)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 31, in is_same_type
    return left.accept(SameTypeVisitor(right))
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 84, in visit_instance
    is_same_types(left.args, self.right.args) and
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 45, in is_same_types
    if not is_same_type(a1[i], a2[i]):
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 28, in is_same_type
    left = simplify_union(left)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 37, in simplify_union
    return make_simplified_union(t.items)
  File "/usr/lib/python3.8/site-packages/mypy/typeops.py", line 366, in make_simplified_union
    if i != j and is_proper_subtype(tj, ti, keep_erased_types=keep_erased):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1177, in _is_proper_subtype
    return left.accept(ProperSubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1273, in visit_instance
    is_protocol_implementation(left, right, proper_subtype=True)):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 556, in is_protocol_implementation
    is_compat = is_proper_subtype(subtype, supertype)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1177, in _is_proper_subtype
    return left.accept(ProperSubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 1098, in accept
    return visitor.visit_callable_type(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1294, in visit_callable_type
    return is_callable_compatible(left, right, is_compat=self._is_proper_subtype)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 857, in is_callable_compatible
    if not ignore_return and not is_compat_return(left.ret_type, right.ret_type):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1207, in _is_proper_subtype
    return is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1177, in _is_proper_subtype
    return left.accept(ProperSubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1267, in visit_instance
    nominal = all(check_argument(ta, ra, tvar.variance) for ta, ra, tvar in
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1267, in <genexpr>
    nominal = all(check_argument(ta, ra, tvar.variance) for ta, ra, tvar in
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1255, in check_argument
    return self._is_proper_subtype(leftarg, rightarg)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1207, in _is_proper_subtype
    return is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1177, in _is_proper_subtype
    return left.accept(ProperSubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 1724, in accept
    return visitor.visit_union_type(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1356, in visit_union_type
    return all([self._is_proper_subtype(item, self.orig_right) for item in left.items])
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1356, in <listcomp>
    return all([self._is_proper_subtype(item, self.orig_right) for item in left.items])
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1207, in _is_proper_subtype
    return is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1172, in _is_proper_subtype
    return any([is_proper_subtype(orig_left, item,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1172, in <listcomp>
    return any([is_proper_subtype(orig_left, item,

# Start of recursion

  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1177, in _is_proper_subtype
    return left.accept(ProperSubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1273, in visit_instance
    is_protocol_implementation(left, right, proper_subtype=True)):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 527, in is_protocol_implementation
    if (mypy.sametypes.is_same_type(l, left)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 31, in is_same_type
    return left.accept(SameTypeVisitor(right))
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 84, in visit_instance
    is_same_types(left.args, self.right.args) and
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 45, in is_same_types
    if not is_same_type(a1[i], a2[i]):
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 28, in is_same_type
    left = simplify_union(left)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 37, in simplify_union
    return make_simplified_union(t.items)
  File "/usr/lib/python3.8/site-packages/mypy/typeops.py", line 366, in make_simplified_union
    if i != j and is_proper_subtype(tj, ti, keep_erased_types=keep_erased):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1177, in _is_proper_subtype
    return left.accept(ProperSubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1273, in visit_instance
    is_protocol_implementation(left, right, proper_subtype=True)):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 527, in is_protocol_implementation
    if (mypy.sametypes.is_same_type(l, left)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 31, in is_same_type
    return left.accept(SameTypeVisitor(right))
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 84, in visit_instance
    is_same_types(left.args, self.right.args) and
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 45, in is_same_types
    if not is_same_type(a1[i], a2[i]):
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 28, in is_same_type
    left = simplify_union(left)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 37, in simplify_union
    return make_simplified_union(t.items)
  File "/usr/lib/python3.8/site-packages/mypy/typeops.py", line 366, in make_simplified_union
    if i != j and is_proper_subtype(tj, ti, keep_erased_types=keep_erased):

# Removed 30000 duplicate messages

  File "/usr/lib/python3.8/site-packages/mypy/typeops.py", line 366, in make_simplified_union
    if i != j and is_proper_subtype(tj, ti, keep_erased_types=keep_erased):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1177, in _is_proper_subtype
    return left.accept(ProperSubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/typing.py", line 873, in __new__
    obj = super().__new__(cls)
RecursionError: maximum recursion depth exceeded while calling a Python object

To Reproduce

Playground. However, I'm not sure if these are the same errors. My local mypy results in a status 2 where the playground has status 139.

Existing issues

This error seems related to #6730 however the tracebacks are different. Whilst the crashes may be related I think this shows that the current code is exponential leading to crashes.

My recursive traceback
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1177, in _is_proper_subtype
    return left.accept(ProperSubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1273, in visit_instance
    is_protocol_implementation(left, right, proper_subtype=True)):
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 527, in is_protocol_implementation
    if (mypy.sametypes.is_same_type(l, left)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 31, in is_same_type
    return left.accept(SameTypeVisitor(right))
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 84, in visit_instance
    is_same_types(left.args, self.right.args) and
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 45, in is_same_types
    if not is_same_type(a1[i], a2[i]):
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 28, in is_same_type
    left = simplify_union(left)
  File "/usr/lib/python3.8/site-packages/mypy/sametypes.py", line 37, in simplify_union
    return make_simplified_union(t.items)
  File "/usr/lib/python3.8/site-packages/mypy/typeops.py", line 366, in make_simplified_union
    if i != j and is_proper_subtype(tj, ti, keep_erased_types=keep_erased):
6730 recursive traceback
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1156, in is_proper_subtype
    return _is_proper_subtype(left, right,
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1177, in _is_proper_subtype
    return left.accept(ProperSubtypeVisitor(orig_right,
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/subtypes.py", line 1261, in visit_instance
    left = map_instance_to_supertype(left, right.type)
  File "/usr/lib/python3.8/site-packages/mypy/maptype.py", line 24, in map_instance_to_supertype
    return map_instance_to_supertypes(instance, superclass)[0]
  File "/usr/lib/python3.8/site-packages/mypy/maptype.py", line 37, in map_instance_to_supertypes
    a.extend(map_instance_to_direct_supertypes(t, sup))
  File "/usr/lib/python3.8/site-packages/mypy/maptype.py", line 82, in map_instance_to_direct_supertypes
    t = expand_type(b, env)
  File "/usr/lib/python3.8/site-packages/mypy/expandtype.py", line 16, in expand_type
    return typ.accept(ExpandTypeVisitor(env))
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 794, in accept
    return visitor.visit_instance(self)
  File "/usr/lib/python3.8/site-packages/mypy/expandtype.py", line 83, in visit_instance
    args = self.expand_types(t.args)
  File "/usr/lib/python3.8/site-packages/mypy/expandtype.py", line 143, in expand_types
    a.append(t.accept(self))
  File "/usr/lib/python3.8/site-packages/mypy/types.py", line 1724, in accept
    return visitor.visit_union_type(self)
  File "/usr/lib/python3.8/site-packages/mypy/expandtype.py", line 123, in visit_union_type
    return make_simplified_union(self.expand_types(t.items), t.line, t.column)
  File "/usr/lib/python3.8/site-packages/mypy/typeops.py", line 366, in make_simplified_union
    if i != j and is_proper_subtype(tj, ti, keep_erased_types=keep_erased):

Caching

The error seems somewhat tempermental. If bar is proceeded with a subset of the type it seems some caching comes in to play. This runs without error.

from typing import Union, Iterator, Iterable


def foo(
    values: Union[
        Iterator[Union[Iterator[int], Iterable[int]]],
        Iterable[Union[Iterator[int], Iterable[int]]],
    ]
) -> Iterator[int]:
    for i in values:
        for j in i:
            yield j


def bar(
    values: Union[
        Iterator[Union[
            Iterator[Union[Iterator[int], Iterable[int]]],
            Iterable[Union[Iterator[int], Iterable[int]]],
        ]],
        Iterable[Union[
            Iterator[Union[Iterator[int], Iterable[int]]],
            Iterable[Union[Iterator[int], Iterable[int]]],
        ]],
    ]
) -> Iterator[int]:
    for i in values:
        for j in i:
            for k in j:
                yield k


foo([[1]])
bar([[[1]]])
@hauntsaninja
Copy link
Collaborator

That's fun; I can repro. Thanks for the clear report.

@JelleZijlstra
Copy link
Member

Not that this isn't a bug, but note that Iterator is a subclass of Iterable (https://github.com/python/typeshed/blob/master/stdlib/3/typing.pyi#L177; every Iterator is also an Iterable), so the Union in your original example is redundant.

@Peilonrayz
Copy link
Contributor Author

Thank you @JelleZijlstra. If I had checked collections.abc I would have seen this too, I feel like a bit of a numpty now. Thank you I'll shift over to that.

Tests to check Iterable = Iter
def foo_list() -> Iterable[int]:
    return [0]


def foo_set() -> Iterable[int]:
    return {0}


def foo_dict() -> Iterable[int]:
    return {0: 'a'}


def foo_iterator() -> Iterable[int]:
    yield 0


def bar(foo: Iterable[int]) -> None:
    pass


bar([0])
bar({0})
bar({0: 'a'})
bar(iter([0]))

In light of this I tested changing to Iter = Union[Iterable[T], Iterator[T]]. The code runs fine. I'd speculate that recursion is being used to iterate through the origins. (To be clear this is just to help understand the bug.)

from typing import Union, Iterator, Iterable


def bar(
    values: Union[
        Iterable[Union[
            Iterable[Union[Iterable[int], Iterator[int]]],
            Iterator[Union[Iterable[int], Iterator[int]]],
        ]],
        Iterator[Union[
            Iterable[Union[Iterable[int], Iterator[int]]],
            Iterator[Union[Iterable[int], Iterator[int]]],
        ]],
    ]
) -> Iterator[int]:
    for i in values:
        for j in i:
            for k in j:
                yield k


bar([[[1]]])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants