Completion for symbols including colons (":")

[clason]

I'm using (omni)completion on symbols that include colons (such as eq:strong-convergence), which seems to trip up the logic introduced here in the following situation:

1. Type the completion trigger, in my case \eqref{
2. Type a later part of the symbol after the colon, here eq:conv
3. Trigger completion (either by <c-x c-o> or an autocomplete plugin like ncm2-vim-lsp); the candidate eq:strong-convergence` is shown in the pum.
4. Select this candidate by <c-n>

Instead of replacing the full text, only the text after the colon is replaced. (It works correctly if I type a full initial match after the colon like eq:str or omit the colon like conv.)

(This is with texlab language server for LaTeX, but I don't think this is important here.)

EDIT: This seems to be a general issue with symbol parsing -- I have the same issue with renaming. It seems that the colon is hard-coded as a symbol separator, but this is not necessarily a correct assumption for all languages (in this case, it's definitely wrong for TeX).

[thomasfaingnaert]

@clason
Does something like this work?

augroup latex_iskeyword
    autocmd!
    autocmd FileType tex setlocal iskeyword+=:
augroup end

This will also change the meaning of a word though, so w and iw are changed as well, which may or may not be what you want.

[clason]

@thomasfaingnaert That should already be set by vimtex (https://github.com/lervag/vimtex/blob/8b8684da10c17af4ab449e1e7435d3f9c2459bf8/autoload/vimtex.vim#L386). Is this being overwritten by vim-lsp?

EDIT: It also doesn't fix the issue, I'm afraid.

[thomasfaingnaert]

@clason I'm not sure, what does set iskeyword? say in a tex file?

[clason]

• with vimtex and vim-lsp loaded (in this order):
  iskeyword=@,48-57,_,192-255
• only vimtex:
  iskeyword=@,48-57,_,192-255,:
• with vim-lsp and vimtex loaded (in this order):
  iskeyword=@,48-57,_,192-255,:
  but LSP omnicomplete doesn't work anymore :(

[thomasfaingnaert]

@clason
So I just tested this with a simple file:

\documentclass{article}

\begin{document}
\section{Foo bar}%
\label{sec:foo_bar}
\ref{sec:bar
\end{document}

Invoking C-x C-o at \ref{sec:bar, which results in \ref{sec:barsec:foo_bar} instead of \ref{sec:foo_bar}.
The relevant part in the log is:

Thu 27 Jun 2019 15:19:36 CEST:["--->",1,"texlab",{"method":"textDocument/completion","on_notification":"---funcref---","params":{"textDocument":{"uri":"file:///home/thomas/test.tex"},"position":{"character":12,"line":5}}}]
Thu 27 Jun 2019 15:19:36 CEST:["<---",1,"texlab",{"response":{"id":41,"jsonrpc":"2.0","result":{"isIncomplete":true,"items":[{"label":"sec:foo_bar","data":"Label","kind":5}]}},"request":{"id":41,"jsonrpc":"2.0","method":"textDocument/completion","params":{"textDocument":{"uri":"file:///home/thomas/test.tex"},"position":{"character":12,"line":5}}}}]

So texlab says to vim-lsp: insert the text sec:foo_bar at the current character position literally. Can you test this in VSCode as well? I believe that should have the same result.

[clason]

VS Code behaves slightly differently since it only provides autocomplete, but also incorrectly (completing to sec:sec:foo_bar). So it's indeed a texlab problem, and I'll report it there. Thanks for tracking it down, and sorry for blaming vim-lsp!

For reference, what should have been the correct response in this case? Just a different position?

[thomasfaingnaert]

@clason Returning labels on completion items is still a remnant from the old LSP protocol, and it doesn't work that great in all cases. The problem is that this label can be interpreted differently depending on the lsp client. E.g. suppose I type cons and I invoke completion, and the language server returns 'console. Should the client remove the prefix and just add ole or type console again?

You may want to ask the author of texlab to look into text-edits, which solve these problems nicely.

[clason]

I have one remaining issue with this that is due to vim-lsp (it doesn't occur on VS Code with the language server nor with default omnicomplete): The character range information seems to be thrown off by non-English characters. If I modify your minimal example to

\documentclass{article}

\begin{document}
\section{Foo bar}%
\label{sec:foo_bar}
ö \ref{sec:foo
\end{document}

and select the completion, I get {sec:sec:foo_bar} again. The relevant part of the log is

Thu 04 Jul 2019 04:50:16 PM CEST:["--->", 5, "texlab", {"method": "textDocument/           completion", "on_notification": "---funcref---", "params": {"textDocument": {"uri":        "file:///home/clason/test.tex"}, "position": {"character": 23, "line": 5}}}]
Thu 04 Jul 2019 04:50:16 PM CEST:["<---", 5, "texlab", {"response": {"id": 10, "jsonrpc":  "2.0", "result": {"isIncomplete": true, "items": [{"label": "sec:foo_bar", "data":         "Label", "textEdit": {"range": {"end": {"character": 23, "line": 5}, "start":              {"character": 23, "line": 5}}, "newText": "sec:foo_bar"}, "kind": 1}]}}, "request": {"id": 10, "jsonrpc": "2.0", "method": "textDocument/completion", "params": {"textDocument":      {"uri": "file:///home/clason/test.tex"}, "position": {"character": 23, "line": 5}}}}]

In comparison, without the ö , the log entries are

Thu 04 Jul 2019 05:14:24 PM CEST:["--->", 5, "texlab", {"method": "textDocument/           completion", "on_notification": "---funcref---", "params": {"textDocument": {"uri":        "file:///home/clason/test.tex"}, "position": {"character": 16, "line": 5}}}]               
Thu 04 Jul 2019 05:14:24 PM CEST:["<---", 5, "texlab", {"response": {"id": 9, "jsonrpc":   "2.0", "result": {"isIncomplete": true, "items": [{"label": "sec:foo_bar", "data":         "Label", "textEdit": {"range": {"end": {"character": 16, "line": 5}, "start":              {"character": 5, "line": 5}}, "newText": "sec:foo_bar"}, "kind": 1}]}}, "request": {"id":  9, "jsonrpc": "2.0", "method": "textDocument/completion", "params": {"textDocument":       {"uri": "file:///home/clason/test.tex"}, "position": {"character": 16, "line": 5}}}}] 

This looks like a deep bug, since I can trace at least some intermittent issues I've been having with vim-lsp not showing all the expected completions and hover not working on part of the symbol to the presence of such characters.

[thomasfaingnaert]

@clason
I can reproduce this as well. Interestingly, if I open the file with the ö already present, completion seems to work.
This might be an issue with the updating of the buffer contents using UTF8 characters.

[clason]

@thomasfaingnaert Hmm, this doesn't make a difference for me (to get a minimal log, I opened the minimal example as a previously saved test.tex file). My fileencoding is utf-8, if this helps?

[thomasfaingnaert]

@clason
Nevermind, I think the text edits were not triggered because apparently Vim does not execute CompleteDone if only one match is present and menuone is not in completeopt.

[thomasfaingnaert]

Could you check in VScode if

o \ref{sec:foo

and

ö \ref{sec:foo

send different completion request positions? It seems that Vim counts ö as 2 characters, at least using col(). My guess it that LSP requires this to be counted as one char.

[thomasfaingnaert]

Might be related to #282.

[clason]

It's a bit difficult, since vs code only triggers completion after typing a character, while omnicomplete doesn't require that. With that in mind,

• vim-lsp:

Thu 04 Jul 2019 05:47:27 PM CEST:["<---", 5, "texlab", {"response": {"id": 9, "jsonrpc":   "2.0", "result": {"isIncomplete": true, "items": [{"label": "sec:foo_bar", "data":         "Label", "textEdit": {"range": {"end": {"character": 8, "line": 5}, "start": {"character": 7, "line": 5}}, "newText": "sec:foo_bar"}, "kind": 1}]}}, "request": {"id": 9, "jsonrpc":  "2.0", "method": "textDocument/completion", "params": {"textDocument": {"uri": "file:///   home/clason/test.tex"}, "position": {"character": 8, "line": 5}}}}]                        

• VS Code (or rather, texlab from VS Code)

[2019-07-04 17:47:52] {"jsonrpc":"2.0","result":{"isIncomplete":true,"items":[{"data":     "Label","kind":5,"label":"sec:foo_bar","textEdit":{"newText":"sec:foo_bar","range":{"end": {"character":8,"line":5},"start":{"character":7,"line":5}}}}]},"id":3}Content-Length: 213

• VS Code without ö :

[2019-07-04 17:55:25] {"jsonrpc":"2.0","result":{"data":"Label","insertTextFormat":1,      "kind":5,"label":"sec:foo_bar","textEdit":{"newText":"sec:foo_bar","range":{"end":         {"character":6,"line":5},"start":{"character":5,"line":5}}}},"id":4}Content-Length: 116

[clason]

@thomasfaingnaert Yes, that sounds related; although the PR #284 doesn't fix this, unfortunately.

[thomasfaingnaert]

I'm afraid I'll not be of much help on this issue. Maybe @mattn can help diagnose the problem?

[clason]

That would be much appreciated. Possibly the progress in neovim/neovim#10222 can be of help, too.

[clason]

Maybe I should open a new issue for that -- this is easy enough to reproduce in Python?
(And actually, you can even break vim-lsp this way...)

[clason]

Done: #425