Dialyzing "checklist"

[yrashk]

I am trying to get elixir built into a PLT and I am running into various issues. Let me list them here so we can go through them and fix'em.

• -Build elixir with debug_info by default #580- (closed)
• Processing Macro beam gets dialyzer stuck. I am going to run this again for a night, but last time it took all night and didn't finish.

I can't quite spot what's happening there. The only thing that gets my attention in this module is ladders of "orelse"s and such (try running rp(beam_lib:chunks(Elixir-Macro, [abstract_code])) in erlang shell and you'll see what I mean)

• A good bunch of modules (at least Node, Protocol, Record, Regex, RuntimeError, String, elixir, elixir_aliases, elixir_compiler, elixir_dispatch, elixir_errors, elixir_glob, elixir_import, elixir_macros, elixir_module, elixir_quote, elixir_scope, elixir_sup, elixir_translator, elixir_tree_helpers,elixir_try) seem to make dialyzer [soft]-crash with this warning:

  Adding information to /Users/yrashk/.dialyzer_plt...{"init terminating in do_boot",{function_clause,[{dialyzer,format_warning,
[{warn_callgraph,{<<55 bytes>>,92},{call_to_missing,[erlang,spawn,5]}},basename],[{file,"dialyzer.erl"},{line,292}]},
{dialyzer_cl,'-print_warnings/1-lc$^0/1-0-',2,[{file,"dialyzer_cl.erl"},{line,781}]},{dialyzer_cl,print_warnings,1,
[{file,"dialyzer_cl.erl"},{line,781}]},{dialyzer_cl,return_value,2,[{file,"dialyzer_cl.erl"},{line,667}]},{dialyzer_cl,do_analysis,4,
[{file,"dialyzer_cl.erl"},{line,405}]},{dialyzer,'-cl/1-fun-0-',1,[{file,"dialyzer.erl"},{line,152}]},{dialyzer,doit,1,[{file,"dialyzer.erl"},
{line,243}]},{dialyzer,plain_cl,0,[{file,"dialyzer.erl"},{line,83}]}]}}

Seemingly (according to --plt_info) these modules still make it into PLT. Here's an email I sent to Kostis (@kostis) with the explanation of this problem:

I found an issue that is seemingly preventing me from adding a bunch
of modules to plt. The issue that one of the dialyzer aux functions
crashes with function_clause:

{"init terminating in
do_boot",{function_clause,[{dialyzer,format_warning,[{warn_callgraph,{<<55
bytes>>,92},{call_to_missing,[erlang,spawn,5]}},basename],[{file,"dialyzer.erl"},{line,292}]}

When I started investigating I matched that <<55 bytes>> piece with
the File variable below:

format_warning({_Tag, {File, Line}, Msg}, FOpt) when is_list(File),
                                                     is_integer(Line) ->

However, according to format_warning's spec, the first argument is
dial_warning(), and according to dialyzer.hrl, this is
{dial_warn_tag(), file_line(), {atom(), [term()]}} and file_line() is
-type file_line() :: {file:filename(), non_neg_integer()}.

Which means, the filename accepted should be file:filename(), which is
currently (according to file.erl) is string() | binary()

Therefore, format_warning requiring the list there is in the wrong, if
I am not mistaken.

I wonder, however, why that bit got binarized even in pure .erl files? Something about our compilation process?

So far these seem to be the only obstacle. #1 is trivially fixable (I have a patch if necessary), #3 is seemingly minor as well (but we better figure out why it gets to that warning in the first place (spawn/5?!) and why File gets binarized there). #2 is a completely mystery to me so far.

I'd love if @kostis could help us but I know he is a busy man :)

[yrashk]

#3 mystery resolved, this was incorrect call to spawn/5 in node.ex:92 — the only issue sort of left is the one I described to @kostis. May be they will fix that to be in line with the file:filename() type, may be not.. who knows...

[josevalim]

@yrashk If in fact this is elixir outputting binary where we could output a char list, we could also swap to outputting the char list in the abs tree. i am fine with that.

[yrashk]

I think char list will be safer, as long as what we get back in elixir is a
binary?..

On Sat, Oct 27, 2012 at 2:20 AM, José Valim notifications@github.comwrote:

@yrashk https://github.com/yrashk If in fact this is elixir outputting
binary where we could output a char list, we could also swap to outputting
the char list in the abs tree. i am fine with that.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/581#issuecomment-9833940.

[yrashk]

Current discovery: dialyzer seemingly can't handle "large" guard expressions (say 15-16+ orelse exprs make it spin for some long timer, with seemingly exponential growth)

[yrashk]

This code seems to make dialyzer crunch for a looong time (and longer if you add more clauses): https://gist.github.com/e07da354506aedcccd0c

[kostis]

We can definitely change the relevant guard but I want to understand what's happening first because I am a bit uncertain about what you are doing. For that, I want some more information. Unless I am mistaken, adding to an existing PLT (say to ~/.dialyzer_plt) should not be generating any warnings other than that there are unknown functions or types. If you want to add the info under ebin_dir to this PLT you can do it with a command of the form:

dialyzer --add_to_plt ebin_dir

So, I do not see why you get the crash you are reporting.because warnings should not be produced by this command. What are you using and how can I reproduce this?

While I am writing all these more comments were added above. I would appreciate an exact sequence of commands to run to reproduce these also.

Unrelated personal info: My reaction may not be so fast since I just got a child! -- our first...

[yrashk]

First of all, my congratulations with the birth of the baby! 👶

1. That warning crash. I believe it happens because the file name we embed into our .beam files is a binary, not a character list. This would explain those soft crashes (which we have fixed since)

2. On a more serious note, we suspect dialyzer has a bug or a big inefficiency in handling longer guard clauses: this https://gist.github.com/e07da354506aedcccd0c (and larger sequences) make dialyzer spin for A LONG time.. This is much more serious than (1) which would be nice to get fixed, too, of course — either on our side or dialyzer's

[yrashk]

fyi @josevalim — with our latest workaround (3c15bcc) I successfully PLTd the whole elixir.

[yrashk]

What's more interesting, if I change orelse to ; in that gist, dialyzer has no problem with it. @kostis, might it be that dialyzer misinterprets orelse in some way?

[yrashk]

It'd be fair to note here that ; and orelse generate some vastly different ASTs

[yrashk]

anyhow, that is still quite valid erlang code, I don't think dialyzer should choke on it :)