Handle autocomplete better in macros accepting code blocks

[lf-]

Related: #6097, #7402

I was writing some code recently using tokio::select! and, separately, proptest::proptest!, which are both macro_rules! macros, and the developer experience for these is quite poor: autocomplete simply does not work inside them. I understand that mostly macros are a quagmire for making completion work properly, but these seem like they could be different than the usual case.

The reason that these macros are interesting as a more interesting target for improving autocomplete behaviour than the other cases is that they are both accepting large blocks of code that are passed through unmodified which are most likely where most editing is happening and where it is most troublesome that completion doesn't work. Further, they are (mostly) declarative macros [the proc macro bits of tokio::select! appear to be limited enough that I think this is true] in the parts that deal with the code in the select branches.

Illustrative code example:

    loop {
        tokio::select! {
            v = cap.select_next_some() => {
                let (v, meta) = v?;

                // Completion does not work in here, but maybe it could
            }
            s = tokio::signal::ctrl_c() => {
                let _ = s?;

                break Ok(());
            }
        };
    }

[Veykril]

We should be able to handle this better yes. The reason for why we fail is not clear to me right now (and given the macro is highly recursive its tough to tell at a glance, having a step through macro epansion in r-a would be nice right now).
I imagine the expansions differ slightly (that is the plain macro expansion and the one with a fake ident inserted) causing us to bail out which prevents completions as we are still stuck in a token tree.

#11059 might be related here as the macro calls at some point capture by opaque fragment kinds.

[flodiebold]

As far as I can see it's not so much that the expansions differ, but rather that e.g.

        tokio::select! {
            v = foo() => {
                return v.
            }
        };

fails to expand completely at all. Probably because the block doesn't parse as an expr and our error recovery then probably doesn't pick the right branch (there are so many to choose from). Maybe this could be handled with some better error recovery heuristics (currently we just take the branch that matched more tokens and more metavariables), but... it's probably complicated. A stepwise expansion function would indeed be helpful to see what branch it actually does take, because the expansion I get in the end looks like this:

$crate::select!(@{
  start = {
    $crate::macros::support::thread_rng_n(BRANCHES)
  };
  (_)()v = (foo()),if true => {
    return v.
  }, ;
  panic!("all branches are disabled and there is no else branch");
  [... zip 58 repeats of the same panic call...]
  panic!("all branches are disabled and there is no else branch");
  panic!("all branches are disabled and there is no else branch")
})

@lf- note that (at least in my test) completion works as soon as you type another letter so it works syntactically again, i.e. v.f should give completions.

[lf-]

Yeah I tested some more and confirmed your result. It's still not great because triggering completion for all possible methods isn't possible.