Skip to content

SEGV from endless loop in check_call in case of isinstance(callee, Instance) #7370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dckc opened this issue Aug 19, 2019 · 2 comments
Closed

SEGV from endless loop in check_call in case of isinstance(callee, Instance) #7370

dckc opened this issue Aug 19, 2019 · 2 comments

Comments

@dckc
Copy link

dckc commented Aug 19, 2019 •
edited
Loading

mypy dumped core while checking some code using spark.sql.Column.
The situation is similar to #6730, but using using PYTHONFAULTHANDLER=1 I got a different stack trace:

Current thread 0x00007fa5cceff740 (most recent call first):
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/types.py", line 113 in __init__
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/types.py", line 751 in __init__
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/types.py", line 825 in __init__
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/types.py", line 893 in copy_modified
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 958 in bind_self
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 344 in analyze_member_var_access
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 193 in analyze_instance_member_access
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 117 in _analyze_member_access
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkmember.py", line 103 in analyze_member_access
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 710 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717 in check_call
  ...

I added assert call_function is not callee before the recursive call

mypy/mypy/checkexpr.py

Line 715 in 7aaa435

return self.check_call(call_function, args, arg_kinds, context, arg_names,

and now I get more useful diagnostics, at least:

tumor_reg_data.py:245: error: "object" has no attribute "toPandas"
...
tumor_reg_data.py:693: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.701
tumor_reg_data.py:693: : note: please use --show-traceback to print a traceback when reporting a bug

Line 693 is the items = ... assignment:

def itemNumOfPath(bc_var: DataFrame,
                  item: str = 'item'):
    digits = func.regexp_extract('concept_path',
                                 r'\\i2b2\\naaccr\\S:[^\\]+\\(\d+)', 1)
    items = bc_var.select(digits.cast('int').alias(item)).dropna().distinct()
    return items.sort(item)

IOU the rest of the code, but it's not straightforward to share just now.

$ mypy --version
mypy 0.701

using mypy --show-traceback I get:

export PATH=$HOME/opt/miniconda3/envs/py2tr/bin:$PATH && python -m doctest tumor_reg_data.py && mypy --strict --show-traceback tumor_reg_data.py
tumor_reg_data.py:249: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.701
Traceback (most recent call last):
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/bin/mypy", line 10, in <module>
    sys.exit(console_entry())
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/__main__.py", line 7, in console_entry
    main(None)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/main.py", line 84, in main
    res = build.build(sources, options, None, flush_errors, fscache)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 163, in build
    result = _build(sources, options, alt_lib_path, flush_errors, fscache)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 218, in _build
    graph = dispatch(sources, manager)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 2461, in dispatch
    process_graph(graph, manager)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 2761, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 2868, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/build.py", line 2020, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checker.py", line 284, in check_first_pass
    self.accept(d)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checker.py", line 395, in accept
    stmt.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 976, in accept
    return visitor.visit_expression_stmt(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checker.py", line 2650, in visit_expression_stmt
    self.expr_checker.accept(s.expr, allow_none_return=True, always_allow_any=True)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1679, in accept
    return visitor.visit_op_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1870, in visit_op_expr
    return self.check_boolean_op(e, e)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 2385, in check_boolean_op
    right_type = self.analyze_cond_branch(right_map, e.right, left_type)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3223, in analyze_cond_branch
    return self.accept(node, type_context=context)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1450, in accept
    return visitor.visit_member_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1766, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1777, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1450, in accept
    return visitor.visit_member_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1766, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1777, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1498, in accept
    return visitor.visit_call_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 247, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 292, in visit_call_expr_inner
    callee_type = self.accept(e.callee, type_context, always_allow_any=True)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1450, in accept
    return visitor.visit_member_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1766, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1777, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1498, in accept
    return visitor.visit_call_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 247, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 292, in visit_call_expr_inner
    callee_type = self.accept(e.callee, type_context, always_allow_any=True)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1450, in accept
    return visitor.visit_member_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1766, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 1777, in analyze_ordinary_member_access
    original_type = self.accept(e.expr)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 3255, in accept
    typ = node.accept(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/nodes.py", line 1498, in accept
    return visitor.visit_call_expr(self)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 247, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 317, in visit_call_expr_inner
    ret_type = self.check_call_expr_with_callee_type(callee_type, e, fullname, object_type)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 661, in check_call_expr_with_callee_type
    object_type=object_type)[0]
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 717, in check_call
    callable_node, arg_messages, callable_name, callee)
  File "/home/dconnolly/opt/miniconda3/envs/py3tr/lib/python3.7/site-packages/mypy/checkexpr.py", line 715, in check_call
    assert call_function is not callee, callee
AssertionError: pyspark.sql.column.Column
tumor_reg_data.py:249: : note: use --pdb to drop into pdb
@ilevkivskyi
Copy link
Member

I think this was fixed by #6655, could you please try a newer mypy version? Feel free to re-open if it is not the case.

@dckc
Copy link
Author

dckc commented Aug 20, 2019

Yes, it's fixed in
https://files.pythonhosted.org/packages/73/4f/ed4fbcee5899b1cefe39ebb191a8fdc0c5a03b68930d3f929449b7d2101a/mypy-0.720-cp37-cp37m-manylinux1_x86_64.whl

FWIW, I tried pip install on (a downloaded copy of) https://github.com/python/mypy/archive/v0.720.zip and I got errors about typeshed being broken. I guess that's not expected to work, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dckc @ilevkivskyi