Performance issue

[lfrodrigues]

In one of my projects (https://github.com/Seedstars/django-react-redux-jwt-base) eslint validation takes 3/4s to run.

Adding eslint-plugin-import triples the time to 12s.

Is there a way to improve run time?

[benmosher]

So far, other than #149, I don't have many ideas, and even that will likely only improve performance by about 50%. So I'd predict you'd still be looking at 8s. I have some other ideas about caching a map of imports/exports on the FS, but that would be a pretty major exercise as well, and it's not obvious (without just doing it) how much it would help.

This plugin is so slow--relative to others, at least--because it must parse each imported file into an AST and collect metadata.

Due to the way ESLint is built (with good reason, see discussion), the file I/O must happen synchronously. My impression is that this is where most of the time is lost.

That said, I can't be sure, so it's risky (in a cost-benefit sense) to attempt any high-effort mission to improve it.

[benmosher]

FWIW: eslint_d with the --cache option takes me from ~32s to ~10s.

Just note that there is a known potential gotcha with keeping the export cache in memory for a long time (#200) so it's not perfect, yet. Will try to get this fixed soon.

[lencioni]

Using the timing env variable (e.g. TIMING=1 eslint .), this is what I'm seeing in a project I am using this plugin in, with only the no-unresolved rule enabled:

Rule                          | Time (ms) | Relative
:-----------------------------|----------:|--------:
import/no-unresolved          | 18563.065 |    40.6%
react/sort-comp               |  1621.646 |     3.5%
no-irregular-whitespace       |  1386.958 |     3.0%
react/prop-types              |  1197.552 |     2.6%
no-implied-eval               |   829.967 |     1.8%
react/jsx-uses-vars           |   765.702 |     1.7%
keyword-spacing               |   765.665 |     1.7%
no-eval                       |   578.730 |     1.3%
semi-spacing                  |   549.684 |     1.2%
no-whitespace-before-property |   477.285 |     1.0%

I think for the no-unresolved rule, adding a cache would likely help a good amount.

[lencioni]

Glancing at the code, I think that even memoizing some stuff in the resolve() path would likely help on full runs like this.

[benmosher]

Just published @lencioni's cache contribution to npm as v1.2.0. My total lint time is down from 37s to 25s. Pretty cool!

Notably, my lint time without the import rules is ~12s, so I'm thinking the extra 12s is basically from re-parsing every file for exports. When I've profiled, the parsing is the most expensive bit.

Not sure where to go from here, could try some sort of content-addressed memoizing wrapper parser. Ideally, every module would be parsed only once.

[benmosher]

Just pushed a bunch of perf improvements to next tag as v1.3.0.

I got down to 17s (just above the time without the plugin at all, something like 14-15s) by removing a bunch of unnecessary node_modules parsing, and adding a memoizing parser.

I'm curious to hear how much improvement you see, @lfrodrigues -- let me know if/when you try it? 😄

[lfrodrigues]

@benmosher thanks for the update, I just tested v1.3.0 and I'm happy to report that run time went down to +-4.3s with import/errors and import/warnings enabled.

On my use case eslint-plugin-import is taking around 0.5s which is a lot better than before!