Neovim seems to interpret positions incorrectly

[Alloyed]

the LSP spec specifies that positions are (line, character) pairs, where the character describes the offset from the start of the line in UTF-16 code units.

neovim seems to be interpreting them as visual columns instead.
Here are some test cases:

local a
------^
	local b
-------^
b = "😬" local c = nil
-----------------^

a() b() c()

here are the values from my language server (line:character):

a = 0:6
b = 2:7
c = 4:17 (edit: should be 4:15)

test case A is a control.
test case B has a tab character, which is one character that represents 8 columns
test case C has an emoji, which should be counted as two utf 16 code units

EDIT: C seems to be a goof on my part (I sent bytes instead of code units), I'll leave it as an example but neovim has the correct behavior here

[autozimu]

I don't get it. Could you please elaborate a bit more? What language server, what test file, what's the expect result, what's the real result you got, etc.

[Alloyed]

the language server is one I'm making right now for the lua language. I don't have a release I can share, but I can include:

• the client -> server log:
  https://hastebin.com/kumimosumi.http
• the server -> client log:
  https://hastebin.com/lokecibeku.http
• the test file is above, but here it is again, hopefully more clearly (test.lua):
  https://hastebin.com/obujofesul.lua
  the place the cursor goes after to find each definition is explained visually with the ---^ lines. ^ means the cursor went here, as intended. X means the cursor went here, incorrectly, and O is where I expected the cursor to land.

[autozimu]

I've tried similar situation with rls as language server, and it doesn't have the described issue. Also, I've verified neovim jumping cursor function, cursor(), can correctly handle tab characters.

Not sure what's the real cause on your side. But I cannot do much without stable reproduction steps.

[Avi-D-coder]

@autozimu If cursor is still used then UTF-8 codeunits/bytes are used to index.
Could/would you ever conform to the protocol 3.0 and use UTF-16 units?

I am conducting a survey to inform the debate over what unit ranges should use in the Language Server Protocol.
The debate is occuring in issue #376.