Add @typescript-eslint/parser and typescript-eslint to user tests

[JoshuaKGoldberg]

Adds a usage of @typescript-eslint/parser's parseAndGenerateServices API with relatively more intensive options (especially moduleResolver). It's based on https://github.com/typescript-eslint/typescript-eslint/blob/a2b6b2e0a1ccdfbd5c76ba3fbcffd94d29f5d2b1/packages/typescript-estree/tests/lib/parse.moduleResolver.placeholder-success.test.ts. Running yarn test -in which tests call to TypeScript APIs- is what's running out of memory.typescript-eslint/typescript-eslint#7088

Also adds typescript-eslint as a more typical repo to clone & type check.

cc @jakebailey - we'd chatted about this a bit.

[jakebailey]

Theoretically, I would think that this is enough to capture everything, right?

(Also, could you to ts-eslint add something that can install the latest TS and check it so you get early warnings too?)

[JoshuaKGoldberg]

Theoretically, I would think that this is enough to capture everything, right?

Our git clone -> yarn install & yarn build isn't what's failing in CI though. It's the yarn test. Does the automation also run tests? (sorry if I missed this, it would explain what you've been trying to convey to me)

(Also, could you to ts-eslint add something that can install the latest TS and check it so you get early warnings too?)

Absolutely: typescript-eslint/typescript-eslint#7093

[jakebailey]

The automation only installs without scripts and runs tsc on each tsconfig.json. It never executes code from within the repo, unless you write a script-based test which would do that. If yarn test takes a large amount of time, I don't think we would add it here, but I would me more interested in why things fail only in test and not otherwise... You're still referring to an OOM, right? Not trying to catch errors in the typescript API?

[JoshuaKGoldberg]

still referring to an OOM, right?

Exactly. I don't know why it's OOMing yet, but at the very least I'm hoping that running a similar bit of code to the offending tests would catch similar issues.

[jakebailey]

Going to merge this to test it; if it breaks I'll send a revert.

[jakebailey]

Unfortunately it looks like this infra isn't able to reproduce the problem, but given it finds type errors it's not super clear to me whether or not it's actually successfully building it or not.

[jakebailey]

Starting #6 / 19: https://github.com/typescript-eslint/typescript-eslint.git
/home/vsts/work/1/s> sudo mount -t tmpfs -o size=4g tmpfs /mnt/ts_downloads
Cloning if absent
Cloning https://github.com/typescript-eslint/typescript-eslint.git into typescript-eslint
Installing packages if absent
/mnt/ts_downloads/typescript-eslint> yarn cache clean --all
yarn cache v1.22.19
success Cleared cache.
Done in 3.65s.

Building with /home/vsts/work/1/s/typescript-main/built/local/tsc.js (old)
Building with /home/vsts/work/1/s/typescript-54640/built/local/tsc.js (new)
Comparing errors
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/typescript-estree/tests/fixtures/invalidFileErrors/tsconfig.extra.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/typescript-estree/tests/fixtures/invalidFileErrors/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/typescript-estree/tests/fixtures/moduleResolver/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/typescript-estree/tests/fixtures/projectFolderIgnoreList/ignoreme/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/typescript-estree/tests/fixtures/projectFolderIgnoreList/includeme/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/typescript-estree/tests/fixtures/projectTrue/nested/tsconfig.json: new = 2, old = 2
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/typescript-estree/tests/fixtures/semanticInfo/badTSConfig/tsconfig.json: new = 1, old = 1
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/typescript-estree/tests/fixtures/semanticInfo/tsconfig.json: new = 1, old = 1
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/typescript-estree/tests/fixtures/simpleProject/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/parser/tests/fixtures/services/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/type-utils/tests/fixtures/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin-internal/tests/fixtures/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin-tslint/tests/fixture-project/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin-tslint/tests/test-project/tsconfig-files.json: new = 2, old = 2
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin-tslint/tests/test-project/tsconfig.json: new = 3, old = 3
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin-tslint/tests/tsconfig.json: new = 10, old = 10
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin/tests/fixtures/tsconfig-withmeta.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin/tests/fixtures/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin/tests/fixtures/tsconfig.noImplicitThis.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin/tests/fixtures/tsconfig.noUncheckedIndexedAccess.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/eslint-plugin/tests/fixtures/unstrict/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/packages/website/tsconfig.json: new = 22, old = 22
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/tests/integration/fixtures/eslint-v6/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/tests/integration/fixtures/typescript-and-tslint-plugins-together/tsconfig.json: new = 0, old = 0
Error counts for https://github.com/typescript-eslint/typescript-eslint/blob/a91bb9e264544fc635a44468c2543b94fa83ac1c/tests/integration/tsconfig.json: new = 0, old = 0