Skip to content

CJK Characters in Week Agenda do not Display Correctly #879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Blacktea0 opened this issue Jan 30, 2025 · 13 comments
Closed

CJK Characters in Week Agenda do not Display Correctly #879

Blacktea0 opened this issue Jan 30, 2025 · 13 comments
Labels
bug Something isn't working

Comments

@Blacktea0
Copy link

Blacktea0 commented Jan 30, 2025

Describe the bug

See screenshot, the Monday and January print as some thing like <d0><c7><c6><da><c8><fd>.

The cause of this issue might be that my computer's operating system is Windows 11, with Chinese set as the default display language, and the default encoding in Windows is GBK. In the Chinese environment, "Wendesday" is displayed as "星期三", so when the GBK-encoded content is displayed directly in nvim, it becomes garbled because nvim uses UTF-8 for parsing.

>>> '星期三'.encode('gbk')
b'\xd0\xc7\xc6\xda\xc8\xfd'

Image

Same issue may also exist in Japanese Windows OS which default encoding is Shift JIS.

Steps to reproduce

  1. Open nvim
  2. Open week agenda
  3. See bug

Expected behavior

It should display the Chinese character properly like:

Week-agenda (W05):
星期一 27 一月 2025
星期二 28 一月 2025
星期三 29 一月 2025
星期四 30 一月 2025
星期五 31 一月 2025
星期六 01 二月 2025
星期日 02 二月 2025

or use English:

Image

Emacs functionality

No response

Minimal init.lua

Same as template

Screenshots and recordings

Image

OS / Distro

Windows 11

Neovim version/commit

NVIM v0.10.2

Additional context

No response

@Blacktea0 Blacktea0 added the bug Something isn't working label Jan 30, 2025
@kristijanhusak
Copy link
Member

kristijanhusak commented Jan 30, 2025

Did you try changing the language inside neovim as suggested here https://nvim-orgmode.github.io/troubleshoot#dates-are-not-in-english ?

@Blacktea0
Copy link
Author

Thanks, after adding that command in init script, it works properly.
Just wonder CJK date time notation would be supported in future?

@kristijanhusak
Copy link
Member

I would address it, but I'm not sure how. I'm not doing any encoding, just formatting the date with the os.date builtin function.

What do you get when you execute this in your command line, without the language setting I mentioned in previous comment?

:lua= os.date('%d %B %Y', os.time())

@Blacktea0
Copy link
Author

31 h<d4><c2> 2025
Image

It seems like a bug of the nvim, the expect output is 31 一月 2025.

@kristijanhusak
Copy link
Member

Judging by this question https://vi.stackexchange.com/questions/27706/chinese-characters-become-unrecognizable-overnight-in-neovim, it might be solvable by setting the fileencoding .

Try this:

  1. Open up agenda without the language settings, it should not print things correctly
  2. do :set modifiable and then setlocal fileencoding=utf-8
  3. See if agenda was updated

If it's not updated, then restart Neovim and do this:

  1. Open up neovim
  2. do set fileencoding=utf-8
  3. Open up agenda
  4. Check if it's printed correctly

@Blacktea0
Copy link
Author

Blacktea0 commented Jan 31, 2025

Both methods take no effect.
But if I save content in the buffer to file and reopen it, nvim can detect the correct file encoding and auto set it to euc-cn, and CJK characters are also printed correctly.

I have tried setting :set fileencoding=euc-cn as well, but nothing changed.

asciicast

@kristijanhusak
Copy link
Member

What is your default encoding value? (:echo &encoding)
If it's not utf-8, can you try to do the same thing as for fileencoding, but use encoding and set it to utf-8 :set encoding=utf-8?

@Blacktea0
Copy link
Author

It is utf-8 and it might not be modified in nvim.

@kristijanhusak
Copy link
Member

Can you try switching to this branch and see if it works? https://github.com/nvim-orgmode/orgmode/tree/fix/encoding

@Blacktea0
Copy link
Author

Problem still exists after switch to that branch:

Image

But I found that if I set :language zh_CN.UTF-8, the problem solved and everything works.

You can reproduce the same problem in your machine if you do :language zh_CN.EUC-CN, which is my default locale. I have reproduced same problem in my WSL.

Image

@kristijanhusak
Copy link
Member

Ah, ok, so you adding that to your configuration fixes everything? I'll add this to the troubleshooting if it fixes it for you.

@Blacktea0
Copy link
Author

Yes, it works for me.

Thanks for your help!

@kristijanhusak
Copy link
Member

Awesome!
Added a troubleshooting section for this issue in docs: https://nvim-orgmode.github.io/troubleshoot#chinese-characters-are-not-displayed-correctly-in-agenda

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants