Partial modules and output optimization

[Davidhanson90]

We have a typescript application and a use a file per class. There are lots of benefits to this the major one being source control conflict reduction.

We compile our app using the --out compiler flag but I think there is room for some serious optimization of the compilers output for modules.

Currently if you have two classes defined in separate files the compile generates the following JavaScript.

var Example;
(function (Example) {
    (function (Nested) {
        (function (Issue) {
            var bar = (function () {
                function bar() {
                }
                return bar;
            })();
            Issue.bar = bar;
        })(Nested.Issue || (Nested.Issue = {}));
        var Issue = Nested.Issue;
    })(Example.Nested || (Example.Nested = {}));
    var Nested = Example.Nested;
})(Example || (Example = {}));

var Example;
(function (Example) {
    (function (Nested) {
        (function (Issue) {
            var foo = (function () {
                function foo() {
                }
                return foo;
            })();
            Issue.foo = foo;
        })(Nested.Issue || (Nested.Issue = {}));
        var Issue = Nested.Issue;
    })(Example.Nested || (Example.Nested = {}));
    var Nested = Example.Nested;
})(Example || (Example = {}));

I suggest a compiler flag that composes all classes/ functions etc under a single module declaration? So the output code would be like the example below. This would dramatically reduce the size of the JavaScript the compiler outputs. I estimate that our app which currently compiles to around 15,000 lines of probably be only be 2000 lines if this were available.

var Example;
(function (Example) {
    (function (Nested) {
        (function (Issue) {
            var bar = (function () {
                function bar() {
                }
                return bar;
            })();
            Issue.bar = bar;
            var foo = (function () {
                function foo() {
                }
                return foo;
            })();
            Issue.foo = foo;
        })(Nested.Issue || (Nested.Issue = {}));
        var Issue = Nested.Issue;
    })(Example.Nested || (Example.Nested = {}));
    var Nested = Example.Nested;
})(Example || (Example = {}));

[RyanCavanaugh]

I estimate that our app which currently compiles to around 15,000 lines of probably be only be 2000 lines if this were available.

I want to make sure I'm understanding the magnitude of your scenario based on these numbers -- if you're saving 10 lines of overhead (a three-level deep module) per input .ts file, that means you have (15,000-2,000) / 10 = 1,300 files. If your output would only be 2,000 lines, that means the average input file is contributing 1.5 lines of code to the output? That seems impossible since even an empty class is 5 lines. What's your breakdown like?

[Davidhanson90]

So we have about 8 projects each represents a module (I'm talking ui module here not js ones etc)

A ui module groups all its dependencies under a TS module. They follow a strict structure.

Module.Name.category.SubCategory so for one of our modules you would find folders which represent a category and subcategories. Some examples

Modules.Foo.Filters
Modules.Foo.Data.Entities
Modules.Foo.Data.Services
Modules.Foo.Data.Mappers

Each of these folders can have alot of files with small classes or functions.

As I said we have 8 Ui modules being built, these are built by different teams and they share some core libraries as well so

Core.Framework.Categories.Subcategories
Core.Presentation.Categories.Subcategories

So overall a consistent hierarchically namespaces app.

But if you add up all the bloat for the module pattern being repeated per file it makes for very fat output.

[RamIdeas]

I once wished for this, but after thinking about it, I realised it would create the possibility of leaked private variables. Consider:

module ns1.ns2 {
    var x = 'used by Class A';
    export class A {
        // ...
    }
}
module ns1.ns2 {
    var x = 'used by Class B';
    export class B {
        // ...
    }
}

By your proposal, merging these two module definitions into one closure for output would share (and overwrite) the variable x between the 2 classes.

var ns1;
(function (ns1) {
    (function (ns2) {
        var x = 'used by Class A';
        var A = (function () {
            function A() {
            }
            return A;
        })();
        ns2.A = A;
        var x = 'used by Class B';
        var B = (function () {
            function B() {
            }
            return B;
        })();
        ns2.B = B;
    })(ns1.ns2 || (ns1.ns2 = {}));
    var ns2 = ns1.ns2;
})(ns1 || (ns1 = {}));

A workaround to allow the modules to be merged is to wrap each initial module's output with an IIFE:

var ns1;
(function (ns1) {
    (function (ns2) {
        (function() {
            var x = 'used by Class A';
            var A = (function () {
                function A() {
                }
                return A;
            })();
            ns2.A = A;
        })();
        (function() {
            var x = 'used by Class B';
            var B = (function () {
                function B() {
                }
                return B;
            })();
            ns2.B = B;
        })();
    })(ns1.ns2 || (ns1.ns2 = {}));
    var ns2 = ns1.ns2;
})(ns1 || (ns1 = {}));

However, I proposed a more concise syntax on codeplex which would create one IIFE per module declaration.

var ns1;
(function (ns1, ns2) {
    var x = 'used by Class A';
    var A = (function () {
        function A() {
        }
        return A;
    })();
    ns2.A = A;
})(ns1 || (ns1 = {}), ns1.ns2 || (ns1.ns2 = {}));

var ns1;
(function (ns1, ns2) {
    var x = 'used by Class B';
    var B = (function () {
        function B() {
        }
        return B;
    })();
    ns2.B = B;
})(ns1 || (ns1 = {}), ns1.ns2 || (ns1.ns2 = {}));

Note: There was a bug with my code on codeplex. For a third namespace, the last line would be:

})(ns1 || (ns1 = {}), ns1.ns2 || (ns1.ns2 = {}), ns1.ns2.ns3 || (ns1.ns2.ns3 = {}));

This could be shrunk further by offering a compiler flag stating that all namespaces have been predefined, manually.

So in one file (which needs to precede all files using modules) I could have:

var ns1;
ns1 = {}
ns1.ns2 = {}
ns1.ns2.ns3 = {}

Then the output of each module could be:

(function (ns1, ns2, ns3) {
    // ...
})(ns1, ns1.ns2, ns1.ns2.ns3);

For what it's worth, I wrote a script that takes a JavaScript file output by tsc.exe and transforms it into cleaner JavaScript. It was originally written in JavaScript, but then I ported it into .NET to use as an IItemTransform. IItemTransforms only get applied when bundling is turned on, which meant minification was turned on. If I removed the minification transform, I could see my neatly generated javascript, even though it was all in one big file. Removing the closures, pointed me towards realigning the indentation of each line to something more reasonable, and made debugging much more pleasant (much more so than sourcemaps which reduce the functionality of Chrome's devtools dramatically).

It would be much more useful to have tsc output a nicer js file. Then I wouldn't have to rely on bundling.

I was, like you, also under the impression that it would at least half the size of my production javascript but after minification, it barely made a difference of a couple kilobytes. This is more an issue of cleaner code (closer on par to the one-to-one TypeScript-to-JavaScript ethos) with a slight efficiency boost.

[Davidhanson90]

Regarding variable conflicts I think this could be picked up easily by the compiler as either an error or a warning.

I do really like the suggestion for cleaner module pattern using IIEF.

[RyanCavanaugh]

It's not totally clear what should qualify as a conflict, even:

module m {
    var x = { n: 3 };
}
module m {
    module x {
        export var n;
    }
    console.log(x.n);
}

Maybe if x were { m: 3 }, that wouldn't be a conflict?

I don't think this would be worthwhile if, in practice, most merged modules ended up creating a conflict. Seeing the effects on a larger codebase would be a good exercise; performing this optimization post-build with a parser wouldn't be too difficult.

[Davidhanson90]

I'd argue that should raise a duplicate identifier error.

Perhaps the use of the word "partial" on modules would be a good fit for this. That way existing code bases would be unaffected and we apply a new rule set in the case of partial modules.

[charlessolar]

Another application for a change like this is for merging AMD file namespaces.

I am really disappointed with typescript's performance here - my example is I have a namespace call it Demo.Library with 2 classes, Foo and Bar

Foo and Bar are each their own file but they both export module Demo.Library - what I WANT to happen is for Foo.ts and Bar.ts to be merged into 1 file, call it Library.js so in my app I can require('Library') and get both.

As it stands now, we have to do require('Foo') and require('Bar') and THEN the Demo.Library namespace exists under the import name. See http://www.typescriptlang.org/Handbook#modules-pitfalls-of-modules

What I propose is a way to notify typescript to merge Foo and Bar similarly to the sample on that site which puts the definitions in 1 file. For large scale apps this is definitely a need, I can't put an entire module into 1 file.

[danquirk]

@Volak you're just asking for multi file external modules yes?

[charlessolar]

@danquirk Not exactly - because I know external modules are meant to be 1 file in a folder structure.
I'm asking for a way to define 1 external module over multiple files.

[danquirk]

yes, that's what I meant, just making sure we're on the same page, see #17