Compilation hangs when using Bluebird promise library

[wtrocki]

Introduction

Passing bluebird promise from top level application to separate node.js module causing typescript compiler to hang until it gets out of memory. Problem is happening with multiple typescript compiler versions.

Using promises within the same npm module do not cause any of this problems.

** Environment **
TypeScript Version: Version 2.4.1
Latest bluebird library:

@types/bluebird: "^3.5.5",
bluebird: "^3.5.0",

Code
Simplified example:

// index.ts (separate npm module)
import * as Promise from 'bluebird';
export function init(test: Promise<object>){
      // Code
}

// App using module
import {init} from 'extmodule';
init(promise);

Expected behavior:
Code should compile.

Actual behavior:
Compilation process hangs with extensive memory and processor usage.

[wtrocki]

This problem may be related with Promise being available globally as native promise. Naming bluebird promise imported variable as Promise may just clash with global name. This is causing typescript compiler to end up with infinite loop until entire process gets out of memory.

Another option may be related with BlueBird type definition:
https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/bluebird/index.d.ts#L71-L76

Progress

• I'm going to create sample application to reproduce problem without domain specific code.
• Try to use different versions of BlueBird types to see if problem was introduced in BlueBird types
• Debug compiler to see what part is responsible for extensive memory usage.

[manojmkhanna]

I just stumbled upon the same problem. Using import * as Bluebird from "bluebird"; also doesn't seem to fix the problem. Is there a better way in solving this?

[manojmkhanna]

I have found a more simpler way to reproduce the bug.

Here's the debug repository https://github.com/manojkhannakm/typescript-bluebird-debug.

The repository has two modules module-a and module-b.

module-a has a class called A that has a method createPromise() which returns a Promise.

module-b uses module-a as a local dependency. module-b has a class called B which extends class A. Class B overrides class A's createPromise() method, this is the reason for the the compilation hang.

When running tsc on module-b, after about 5 mins I get this error.

> cd module-b
> tsc

<--- Last few GCs --->

[8912:00000150D2F48250]   223021 ms: Mark-sweep 1409.3 (1533.5) -> 1409.3 (1533.5) MB, 1748.3 / 0.0
ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000006A6DC9BBE9 <JS Object>
    1: instantiateTypeNoAlias [C:\Users\Manoj Khanna\AppData\Roaming\npm\node_modules\typescript\lib
\tsc.js:~28453] [pc=000001746FFA31F8](this=00000147F3678919 <JS Global Object>,type=000000E45F43CFD1
 <a Type with map 0000016075EF79F9>,mapper=00000127FEF952E9 <JS Function mapper (SharedFunctionInfo
0000036127084DA1)>)
    2: getTypeOfInstantiatedSymbol [C:\Users\Manoj Khanna\AppData\Roaming\n...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

All I can think of is that the compiler is stuck in a loop in finding the definition for bluebird in class B.

On removing @types/bluebird and "strict": true from tsconfig on module-b the compilation is successful.

Also I found that publishing module-a to the npm registry and using it in module-b as a public dependency compiles just fine. So the problem occurs only when using module-a as a local dependency.

[sandersn]

@manojkhannakm can you give more detail on your repro? I tried git clone ...; cd module-a; npm i; cd ../module-b; npm i; tsc but I then get the error "cannot find 'module-a'." This doesn't surprise me too much because I don't see any path mapping in tsconfig. When I change module-b's tsconfig, compilation succeeds.

Here's my modified tsconfig. I'm not sure path mapping is the right thing to represent a local dependency from npm. And I had to remove rootDir to get compilation to succeed as well.

{
  "compilerOptions": {
    "module": "commonjs",
    "target": "es6",
    "baseUrl": ".",
    "paths": {
      "module-a": ["../module-a/src"]
    },
    "outDir": "./dist",
    "strict": true,
    "declaration": true,
    "sourceMap": true
  }
}

[sandersn]

@wtrocki I can't repro the crash with your example either. Note that I changed the first line of app/src/index.ts to import {BaseClass} from 'ts-bluebird-module'; since it couldn't find the module 'ts-module'.

[manojmkhanna]

@sandersn I think you missed running tsc on module-a that's why you got Cannot find module 'module-a'.

Also I just tried again with your tsconfig but the compilation still timeouts on module-b.

Edit:

Got this after about 5mins.

C:/Users/Manoj Khanna/Desktop/typescript-bluebird-debug/module-b master
❯ tsc

<--- Last few GCs --->

[3688:000001C7AB5E8A30]   136990 ms: Mark-sweep 1406.2 (1531.5) -> 1406.2 (1531.5) MB, 1432.1 / 0.0
ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000002DBF781BBE9 <JS Object>
    2: findAncestor [C:\Users\Manoj Khanna\AppData\Roaming\npm\node_modules\typescript\lib\tsc.js:~3
07] [pc=000000A8A062963F](this=000002BA5A40B0D1 <an Object with map 0000027EAACE5D59>,node=000002860
9866499 <a Node with map 0000027EAACF8EF1>,callback=0000022696E153A1 <JS Function (SharedFunctionInf
o 000001F1F32EAE61)>)
    3: isSymbolInScopeOfMappedTypeParameter [C:\Users\Manoj Khanna\AppDat...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

[sandersn]

Thanks, that did it @manojkhannakm

[sandersn]

@manojkhannakm's repro compiles in less than a second with typescript@next. @wtrocki can you try typescript@next and see whether your repro is now fixed too? I never got it to repro on my machine.

[manojmkhanna]

@sandersn I just tried debugging from the scratch again but unfortunately the compilation still timeouts. These are the exact steps I followed.

npm i -g typescript@next

git clone https://github.com/manojkhannakm/typescript-bluebird-debug
cd typescript-bluebird-debug

cd module-a
npm i
npm i typescript@next
tsc

cd ..

cd module-b
npm i
npm i typescript@next
tsc    <-- Compilation still hangs here

[manojmkhanna]

On replacing module-b's tsconfig with your above modified tsconfig, the compilation is successful. But the directory structure of module-b/dist is not as expected.

module-b/dist
    module-a
        src
            index.d.ts
            index.js
            index.js.map
    module-b
        src
            index.d.ts
            index.js
            index.js.map

It should be...

module-b/dist
    index.d.ts
    index.js
    index.js.map

I found out that its because of the tsconfig's paths variable. So I changed it to

"paths": {
    "module-a": ["../module-a/dist"]    //Previously ../module-a/src
}

and tried again, the compilation hangs as usual.

[manojmkhanna]

I just found something interesting now, after compiling module-a if I remove the line /// <reference types="bluebird" /> from the file module-a/dist/index.d.ts everything works just fine.

[sandersn]

Yes, with your tsconfig the project does indeed run out of memory.

The problem is that two copies of Bluebird get referenced, one by module-a and one by module-b. The compiler is supposed to be smart enough to dedupe them if they have the same package version. I'm not sure why that doesn't happen here. @Andy-MS has done some work over the last few releases on deduping. Andy, can you take a look?

After the deduping fails, because Typescript is structural, it should be able to relate the two Bluebird copies structurally. But the structure of Bluebird causes the compiler to run out of memory when comparing Bluebird to itself. I haven't found out why yet. I have the failure under the debugger, so I'll update if I find out.

[sandersn]

I haven't found a specific problem besides "Bluebird is very big and recursive", but I made a fix that gets compile time down to 2.9 seconds. This is still considerably slower than the 1 second with `--noStrictGenericChecks".

I extended the type reference caching improvement to decompose array type arguments. In other words, previously the type reference caching would treat the relation of A<T> the same as A<U> if neither T nor U were constrainted. Now it also treats A<T[]> the same as A<U[]>. This works because Bluebird has lots of methods that return Bluebird<T[]>.

This technique, in fact, generalises to any type reference that is a type argument of a type reference. I'm going to try expanding type references recursively when creating a cache key. The cache keys may get too long, but I think it will be instructive either way.

[sandersn]

@wtrocki @manojkhannakm A couple of recent fixes (one of which is described above) should have fixed this. Can you try our your repros with typescript@next?

[manojmkhanna]

@sandersn Its working fine now using typescript@next. Thank you.