Too slow in large projects

[pixelass]

Description

I am using XO and Ava in a large project. Both packages are too slow to be used. I never actually tried to let them finish. I usually give up after 10+ minutes.
I first thought this might be related to globby: sindresorhus/globby#43

As mentioned in that issue I created a test source that reproduces this issue.
There is only one test file in the test-project.
The glob targets a folder that exists only once.

I will open a separate issue in XO if needed.

Test Source

• https://github.com/pixelass/ava-xo-test
• https://github.com/pixelass/ava-xo-test/blob/master/one/a/b/c/d/e/f/g/h/i/j/this-is-it/tests/one.js

Error Message & Stack Trace

Keeps doing this thing: 🤣

⠧

If using --verbose nothing happens

Config

https://github.com/pixelass/ava-xo-test/blob/master/package.json#L41-L59

{
  "ava": {
    "files": [
      "**/this-is-it/**/tests/**/*.js"
    ],
    "source": [
      "**/this-is-it/**/*.js"
    ],
    "presets": [
      "@ava/stage-4",
      "@ava/transform-test-files"
    ],
    "failFast": true,
    "tap": false,
    "require": [
      "babel-register",
      "ignore-styles"
    ],
    "babel": "inherit"
  }
}

Command-Line Arguments

https://github.com/pixelass/ava-xo-test/blob/master/package.json#L10

{
"scripts": {
    "nyc:ava": "nyc ava",
    "ava": "ava",
    "test": "npm run nyc:ava",
  },
}

ava

Relevant Links

Minimal reproduction:
https://github.com/pixelass/ava-xo-test/

Environment

OS X 10.12.5 (16F73)  
Node 6.11.0  

Requested log:

Platform: darwin 16.6.0
Ava: 0.19.1
node: 6.11.0
npm: 3.10.10

[pixelass]

Maybe related to several other issues?
I hope the minimal project I provided helps.

• related to slow (>1 minute) start up time when pointing to directory containing 15 files #1329
• related to Ava slows when there are many non-run files #1288
• related to Crashes while looking for helper files in huge project subdirectories #1228
• related to Slow when linting a single file xojs/xo#212

[pixelass]

I added a branch which uses stricter paths and should ignore the other files.

https://github.com/pixelass/ava-xo-test/tree/direct-path

Config:

https://github.com/pixelass/ava-xo-test/blob/direct-path/package.json#L43-L61

{
  "files": [
      "./one/a/b/c/d/e/f/g/h/i/j/this-is-it/**/tests/**/*.js"
    ],
    "source": [
      "./one/a/b/c/d/e/f/g/h/i/j/this-is-it/**/*.js"
    ],
}

[pixelass]

related to xojs/xo#234

[afenton90]

I'm also finding AVA way too slow on larger projects.
Especially when combined with Babel.
Although a lot of AVA features are awesome, the performance is crippling on larger projects. The lack of feedback on the progress of https://github.com/avajs/ava/projects/1 is also pretty frustrating and until then these issues (relating to performance) will keep cropping up.
There needs to be one issue where the main performance issues are made clear, at the moment, there seems to be no clear direction or strategy for resolving these issues.

[coderas]

👍 Considering we initially adopted AVA for the speed benefits, these problems are a real concern for us - we may even have to migrate our tests back to mocha https://github.com/jfmengels/mocha-vs-ava-performance

That would be a sad day, since the principles of this project are spot on IMO

[novemberborn]

The lack of feedback on the progress of avajs/ava/projects/1 is also pretty frustrating and until then these issues (relating to performance) will keep cropping up.

You should interpret that as there being no progress. That's frustrating (I'd love that project being finished too!) but there's only so many unpaid hours in a week.

There needs to be one issue where the main performance issues are made clear, at the moment, there seems to be no clear direction or strategy for resolving these issues.

There is a plan when it comes to precompilation performance. If somebody were to summarize the various issue threads so we can close them in favor of one summary issue that would be wonderful.

[pixelass]

That would be a sad day, since the principles of this project are spot on IMO

I love Ava. especially in 'normal' projects.

It just works (mostly 🤣)

We ended up removing Ava from the "big project" that had the issue. Most developers in that project are not used to writing tests anyways, so I'm the only one frowning ;(.

I added Ava to the project because it is so easy to write test and understand the possibilites.
There is basically no need to learn an entire suite to write simple tests. More complex things usually require 2 minutes of looking at the docs. Ava is and remains a great API.

I'm happy to contribute. @novemberborn can you point me to a few or did you mean "search for" and then "summarize" 😇 ?

[novemberborn]

I'm happy to contribute. @novemberborn can you point me to a few or did you mean "search for" and then "summarize" 😇 ?

Thanks @pixelass!

It'll take me a while to do the searching myself (due to time limitations), so if you could start that'd be swell 👍

[pixelass]

Performance issues:

#594
#483
#789
#939
#1329
#1318
#1288
#1095

These are issues either tagged with the label performance.
Also includes isses found via searching: "slow", "speed"

That's a lot of stuff to read before being able to summarize or sort out actual issues/suggestions and/or examples.

How should we continue with this? My attempt would be to evaluate the listed issues and write a new issue that lists the different performance issues.

I'd extract the gist of the problem (description/example/potential reason) and link the originals.
This way those other performance issues could be closed/marked when you're happy with the summary. But I'm actually not sure if thats what you meant.

Sadly I don't have a lot of time myself. The weekend is over so I won't be able to start for another week. I'm also really bad at reading long texts so this may take a while 😛.

I skimmed a few of the issues and realized this might be quite a chunk of work.

Any suggestions on the format are welcome. I'd really hate to spend time on something you can't use.

[OmgImAlexis]

Since xo and ava share a lot of underlying libs this may have some relevance.

Ref: xojs/xo#212

[novemberborn]

How should we continue with this? My attempt would be to evaluate the listed issues and write a new issue that lists the different performance issues.

I'd extract the gist of the problem (description/example/potential reason) and link the originals.
This way those other performance issues could be closed/marked when you're happy with the summary. But I'm actually not sure if thats what you meant.

Sadly I don't have a lot of time myself. The weekend is over so I won't be able to start for another week. I'm also really bad at reading long texts so this may take a while 😛.

I skimmed a few of the issues and realized this might be quite a chunk of work.

Any suggestions on the format are welcome. I'd really hate to spend time on something you can't use.

@pixelass that sounds good. Perhaps you could publish a draft in a gist, so we can discuss it before creating the resulting issue? And no worries regarding time commitments.

[wmertens]

I also believe this is an important issue: #1320 (comment)

Try this:

I just went into the module, prepended the helper globs with my source dirs (e.g. '{src,lib}//tests/helpers//*.js'), and ava run time decreased by 20s!

So in node_modules/ava/lib/ava-files.js I changed the helper patterns to:

const defaultHelperPatterns = () => [
        '{src,lib}/**/__tests__/helpers/**/*.js',
];

you need to adjust the glob for your project of course.

[tomheadifen]

@wmertens To confirm this fixed the issue for me too. Went from about 8 seconds to run my tests to around 500ms.

[dhershman1]

Food for thought in the meantime, I swapped out the babel-register module in favor of the esm module and it dropped my time from about a minute and a half to 20-25 seconds.

My library is a heavy ESM module (import/export everywhere) with ~280 unit tests, this is a tremendous win for me in terms of run time on tests.