Server seems to be scanning more files than I would expect #2849

samueljoli · 2024-09-12T07:51:16Z

How are you using the lua-language-server?

NeoVim

Which OS are you using?

MacOS

What is the issue affecting?

Other

Expected Behaviour

In response to the following warning:

More than 100000 files have been scanned. The current scanned directory is `/development/lakay`.

I've added workspace.ignoreDir config like so:

	lua_ls = {
		settings = {
			Lua = {
				workspace = {
					ignoreDir = { "node_modules", ".git", "dist", "build" },
				},
				completion = {
					callSnippet = "Replace",
				},
				diagnostics = { disable = { "missing-fields" } },
			},
		},
	},

The directory itself doesn't have that many files and the outputted directory in the warning matches the one one in the logs:

  rootPath = "/development/lakay",
  rootUri = "file:///development/lakay",

I'm not sure why I would still be getting this message. I don't think there's an actual bug, but I would love some guidance as to what I might be missing

Actual Behaviour

I continue to receive the same warning.

Reproduction steps

Configure lua_ls

local servers = {
	lua_ls = {
		-- cmd = {...},
		-- filetypes = { ...},
		-- capabilities = {},
		settings = {
			Lua = {
				workspace = {
					ignoreDir = { "node_modules", ".git", "dist", "build" },
				},
				completion = {
					callSnippet = "Replace",
				},
				diagnostics = { disable = { "missing-fields" } },
			},
		},
	},
}

require("mason").setup()

local ensure_installed = vim.tbl_keys(servers or {})
vim.list_extend(ensure_installed, {
	"stylua", -- Used to format Lua code
})
require("mason-tool-installer").setup({ ensure_installed = ensure_installed })

require("mason-lspconfig").setup({
	handlers = {
		function(server_name)
			local server = servers[server_name] or {}
			server.capabilities = vim.tbl_deep_extend("force", {}, capabilities, server.capabilities or {})
			require("lspconfig")[server_name].setup(server)
		end,
	},
})

The text was updated successfully, but these errors were encountered:

tomlau10 · 2024-09-12T10:24:07Z

The directory itself doesn't have that many files and the outputted directory in the warning matches the one one in the logs:

I strongly suspect that there may be other hidden files under your /development/lakay root path. While luals will only preload .lua files, it has to scan the sub-directories recursively to find out those .lua files. And if there are many unrelated files (which are not ignored in workspace.ignoreDir), this will increase the scan time.

Here is a bash command to find out numbers of files in each sub-directories:
https://superuser.com/a/474339

find . -maxdepth 1 -mindepth 1 -type d | while read dir; do
  printf "%-25.25s : " "$dir"
  find "$dir" -type f | wc -l
done

You may need to found out which sub folders have that many files first 😕

samueljoli · 2024-09-12T17:34:13Z

@tomlau10 Thanks for the nifty bash script! Here's the output:

./home-manager            :       28
./.git                    :      558
./.direnv                 :        2

tomlau10 · 2024-09-13T01:27:51Z

That's strange 🤔 Further ideas:

Are there any symbolic link which linked to other directories?
I don't know if luals will follow symbolic link to scan files or not.
Do you have a .luarc.json for your project, and did you specify any workspace.library in it?
luals will also scan those folders recursively
https://luals.github.io/wiki/settings/#workspacelibrary
- this comment lua_lsp is abnormally consuming memory. #2704 (comment) also explains that if you have a lot files in Lua.workspace.library setting, luals will consume a huge memory
- this is the related FAQ: https://luals.github.io/wiki/faq/#how-can-i-improve-startup-speeds

In general luals will scan your workspace folders recursively, and AFAIK the directory of the 1st file that you opened in each neovim session is considered as the workspace.
i.e. if you open a file in root /, luals will search all directories recursively under /

Maybe you can paste your log files here for further debugging, like in this issue #2744 (comment).
The log file should contains something like this:

[23:10:24.537][info] [#0:script\workspace\workspace.lua:324]: Preload start:	file:///c%3A/Users/TRUNG%20NGUYEN/AppData/Local/nvim-data
[23:10:24.545][info] [#0:script\workspace\workspace.lua:330]: Scan files at:	file:///c%3A/Users/TRUNG%20NGUYEN/AppData/Local/nvim-data

(there should be another line for Scan library at:)

This shows the root path that luals starts its preloading and scanning, which is generally the 1st file that you opened with neovim

samueljoli · 2024-09-14T16:07:49Z

Here are the logs for Preload start & Scan files at:

[11:46:06.899][info] [#0:script/workspace/workspace.lua:324]: Preload start:	<fallback>
[11:46:06.902][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/basic.lua
[11:46:06.902][info] [#0:script/library.lua:212]: Init builtin library at:	file:///Users/<user_name>/development/<project_name>
[11:46:06.921][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/builtin.lua
[11:46:06.921][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/coroutine.lua
[11:46:06.921][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/io.lua
[11:46:06.922][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/table.lua
[11:46:06.922][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/utf8.lua
[11:46:06.922][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/debug.lua
[11:46:06.922][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/math.lua
[11:46:06.922][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/string.lua
[11:46:06.922][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/package.lua
[11:46:06.922][info] [#0:script/workspace/workspace.lua:353]: Scan library at:	file:///Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/meta/Lua%205.4%20en-us%20utf8/os.lua
[11:46:07.027][info] [#0:script/workspace/workspace.lua:324]: Preload start:	file:///Users/<user_name>/development/<project_name>
[11:46:07.038][info] [#0:script/workspace/workspace.lua:330]: Scan files at:	file:///Users/<user_name>/development/<project_name>

@tomlau10

Are there any symbolic link which linked to other directories?

Yes! I think this is the culprit

I should mention that I'm experiencing this in my project for my dotfiles managed by home-manager, which uses direnv and that seems to set a symbolic link which points to my nix store. So based on your suspicions, I think lua_ls is pulling in all of derivations.

Here are some other interesting logs:
I'm not sure what this config is, but what's worth looking at is workspace.library

[11:46:06.642][warn] [#0:script/config/loader.lua:116]: No config?	{}
[11:46:06.642][info] [#0:script/provider/provider.lua:61]: Load config from client	fallback
[11:46:06.642][info] [#0:script/provider/provider.lua:62]: nil
[11:46:06.768][info] [#0:script/library.lua:212]: Init builtin library at:	file:///Users/<user_name>/development/<project_name>
[11:46:06.784][info] [#0:script/library.lua:212]: Init builtin library at:	nil
[11:46:06.793][warn] [#0:script/config/loader.lua:116]: No config?	{}
[11:46:06.793][info] [#0:script/provider/provider.lua:61]: Load config from client	fallback
[11:46:06.793][info] [#0:script/provider/provider.lua:62]: nil
[11:46:06.793][info] [#0:script/provider/provider.lua:46]: Load config from client	file:///Users/<user_name>/development/<project_name>
[11:46:06.793][info] [#0:script/provider/provider.lua:47]: {
  Lua = {
    completion = {
      callSnippet = "Replace"
    },
    diagnostics = {
      disable = { "missing-fields" }
    },
    runtime = {
      path = { "?.lua", "?/init.lua" },
      pathStrict = true,
      version = "LuaJIT"
    },
    workspace = {
      checkThirdParty = false,
      ignoreDir = { "/lua" },
      library = { "/nix/store/vwaz1lblb8zqimvf0fi1lr4md2my45iy-neovim-unwrapped-nightly/share/nvim/runtime/lua", "/nix/store/i030h473b8pzqhqilgbjd9ca717h5cix-vimplugin-lua5.1-telescope.nvim-scm-1-unstable-2024-05-03/lua", "/nix/store/g6044drcgm990l70rkqb62kph9rn2rsk-vimplugin-cmp-nvim-lsp-2023-12-10/lua", "/nix/store/5ns7j6ypxvkqzdxdrk8vr0aj4z3gb3qz-vimplugin-mason.nvim-2024-03-21/lua", "/nix/store/34ai2f31v0h7iyl3ixx4dzz905swjkwz-vimplugin-mason-tool-installer.nvim-2024-04-11/lua", "/nix/store/x57ydcp19fz7ppls4l6wi3xx1nh4wbdb-vimplugin-mason-lspconfig.nvim-2024-05-05/lua", "/nix/store/ng4y4gimjl6yl61mhwjl8r6fvshaygqw-vimplugin-nvim-lspconfig-2024-05-06/lua", "/nix/store/qm583s0pv1rzqmpk26m9r5j6cxxci9hs-vimplugin-typescript-tools.nvim-2024-01-16/lua" }
    }
  }
}
[11:46:06.793][warn] [#0:script/config/loader.lua:116]: No config?	{}

This one seems to be pulling in my config the way that I've defined it.

[11:46:06.535][info] [#0:script/language.lua:137]: VSC language: nil
[11:46:06.535][info] [#0:script/language.lua:138]: LS  language: en-us
[11:46:06.535][info] [#0:script/workspace/workspace.lua:38]: Workspace init root: 	file:///Users/<user_name>/development/<project_name>
[11:46:06.535][info] [#0:script/workspace/workspace.lua:42]: Log path: 	/Users/<user_name>/.local/share/nvim/mason/packages/lua-language-server/libexec/log/file_Users_<user_name>_development_<project_name>.log
[11:46:06.536][info] [#0:script/workspace/workspace.lua:48]: Workspace create: 	file:///Users/<user_name>/development/<project_name>
[11:46:06.642][info] [#0:script/provider/provider.lua:46]: Load config from client	file:///Users/<user_name>/development/<project_name>
[11:46:06.642][info] [#0:script/provider/provider.lua:47]: {
  Lua = {
    completion = {
      callSnippet = "Replace"
    },
    diagnostics = {
      disable = { "missing-fields" }
    },
    workspace = {
      ignoreDir = { ".direnv", ".git", ".direnv/flake-inputs", "dist", "build", "node_modules", "flake.nix", "flake.lock" }
    }
  }
}
[11:46:06.642][info] [#0:script/provider/provider.lua:46]: Load config from client	file:///Users/<user_name>/development/<project_name>
[11:46:06.642][info] [#0:script/provider/provider.lua:47]: {
  Lua = {
    completion = {
      callSnippet = "Replace"
    },
    diagnostics = {
      disable = { "missing-fields" }
    },
    workspace = {
      ignoreDir = { ".direnv", ".git", ".direnv/flake-inputs", "dist", "build", "node_modules", "flake.nix", "flake.lock" }
    }
  }
}

Figured I'd add links to nix related concepts in the case that you weren't familiar with nix 😅.

tomlau10 · 2024-09-15T01:27:41Z

Are there any symbolic link which linked to other directories?
Yes! I think this is the culprit

Then I suggest adding that symlink to ignoreDir as well 🤔

I am not familiar with neovim or nix (I use vscode). And I have no idea how those /nix/store/* are included in a seemingly unknown config file. 😇 Maybe as you have said, you have a symlink to the /nix/* folder and inside it there is a luals config file which luals will read it as well.

In addition, I suggest using a .luarc.json / .luarc.jsonc in your project folder to specify luals related configs (in case you did not) for better control. This is the recommended way

Maybe if luals found a setting file at your project root, then it will stop looking for other config files in an unexpected directory.

samueljoli · 2024-09-15T22:29:39Z

Yes .direnv was what I ended up adding as option to ignoreDir.

I added a .luarc.json file as you suggested and that seemed to solve the problem. Thank you very much the help. I really appreciate it!

dkarter · 2025-02-04T07:09:55Z

Thanks for this post! I had the same issue, and it almost drove me crazy - it was a symlink that caused an infinite loop. Figured I'd share to save others a headache.

When I was searching I tried for terms like "Loading workspace lua_ls stuck".

For me it was because of tmux-fingers - I had it build the binary it uses from source, instead of the normal "install binary with brew" option, and it created a symlink folder that links to its parent, causing an infinite loop.

This is how I found all the symlinks in the folder:

find . -type l -not -path "*/node_modules/*"

Output

./config/tmux/plugins/tmux-fingers/lib/tablo/lib
./config/tmux/plugins/tmux-fingers/lib/cling/lib
./config/tmux/plugins/tmux-fingers/lib/xdg_base_directory/lib

Then I checked one of these folders:

❯ ll config/tmux/plugins/tmux-fingers/lib/tablo
Permissions Size User   Group Date Modified Name
drwxr-xr-x@    - dorian staff  3 Feb 09:58   examples/
-lrwxr-xr-x@    - dorian staff  3 Feb 09:58   lib -> ../
drwxr-xr-x@    - dorian staff  3 Feb 09:58   spec/
drwxr-xr-x@    - dorian staff  3 Feb 09:58   src/
.rw-r--r--@  150 dorian staff  3 Feb 09:58   .editorconfig
.rw-r--r--@  182 dorian staff  3 Feb 09:58   .gitignore
.rw-r--r--@   18 dorian staff  3 Feb 09:58   .travis.yml
.rw-r--r--@ 1.1k dorian staff  3 Feb 09:58   LICENSE
.rw-r--r--@  33k dorian staff  3 Feb 09:58  󰂺 README.md
.rw-r--r--@  113 dorian staff  3 Feb 09:58   shard.yml

I solved the issue by deleting the tmux-fingers folder and re-installing the plugin.

rm -rf config/tmux/plugins/tmux-fingers

This was referenced Sep 15, 2024

Fix luals memory issue samueljoli/lakay#17

Closed

fix(neovim): fixes lua_ls memory issue samueljoli/lakay#19

Merged

samueljoli closed this as completed Sep 16, 2024

tomlau10 mentioned this issue Nov 8, 2024

Lua_ls is taking 1 666 mb of ram!!!!!! #2940

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Server seems to be scanning more files than I would expect #2849

Server seems to be scanning more files than I would expect #2849

samueljoli commented Sep 12, 2024 •

edited

Loading

tomlau10 commented Sep 12, 2024

samueljoli commented Sep 12, 2024

tomlau10 commented Sep 13, 2024

samueljoli commented Sep 14, 2024 •

edited

Loading

tomlau10 commented Sep 15, 2024

samueljoli commented Sep 15, 2024 •

edited

Loading

dkarter commented Feb 4, 2025

Server seems to be scanning more files than I would expect #2849

Server seems to be scanning more files than I would expect #2849

Comments

samueljoli commented Sep 12, 2024 • edited Loading

How are you using the lua-language-server?

Which OS are you using?

What is the issue affecting?

Expected Behaviour

Actual Behaviour

Reproduction steps

tomlau10 commented Sep 12, 2024

samueljoli commented Sep 12, 2024

tomlau10 commented Sep 13, 2024

samueljoli commented Sep 14, 2024 • edited Loading

tomlau10 commented Sep 15, 2024

samueljoli commented Sep 15, 2024 • edited Loading

dkarter commented Feb 4, 2025

samueljoli commented Sep 12, 2024 •

edited

Loading

samueljoli commented Sep 14, 2024 •

edited

Loading

samueljoli commented Sep 15, 2024 •

edited

Loading