Typescript aborts on shebang.

[jdfreder]

The compiler should allow shebangs to pass through.

http://stackoverflow.com/questions/23298295/how-to-make-a-shell-executable-node-file-using-typescript

[CyrusNajmabadi]

I'm confused. Why would this be part of the typescript compiler? As your link shows, the more appropriate thing to do would be to add the script directive as part of the compilation pipeline with another tool.

[mhegazy]

cause ppl like to write their shell scripts in ts :)

[jdfreder]

Two reasons:

1. Even though shebangs may not be official Javascript spec, I'd expect Typescript to support Node.js Javascript fully, since Typescript is a Javascript superset and Node.js is popular Javascript platform. The following is parsed by node, but not Typescript.

#!/usr/bin/env node
console.log('Hello world.');

2. Is it not easier to ignore the shebang in the Typescript compiler (only one in existence), than it is to add kludge compilation step that prepends the shebang (for N projects that use Typescript for node.js CLIs)? ... I guess you could counter argue N is small, but I think it's actually big.

[jdfreder]

cause ppl like to write their shell scripts in ts :)

+1 : )

[tomlarkworthy]

+1 it be nice to build fully functioning shell scripts written in Typescript like I do with Python or Bash or Javascript, without involving an additional build step.

[poelstra]

+1

I would very much like not having to type "node ..." in front of every TS CLI util / 'shell script', or having to create an almost-empty .js wrapper for them.

@CyrusNajmabadi: arguments for handling it in the compiler:

• compile-on-save tools like Visual Studio, Atom Typescript or Sublimetext cannot be used on these files
• it's not as easy as simply catting a line in front of it: it also e.g. involves updating the sourcemap
• it either requires some kind of naming convention, or a manually maintained list of files that would need this transformation in the 'makefile'. I find such lists to be cumbersome, typically leading to unnecessary merge conflicts (like a Visual Studio project, or tsconfig's files property).

I filed this on codeplex quite a while back, and apparently so did this user.

As noted in my codeplex issue though, there are some things to take into consideration:

• it's a bit dubious whether having #!/bin/env node at the top of a .ts file actually makes sense (because Node cannot run the .ts file directly). This is in contrast to e.g. Javascript or Python files, which can actually be directly run by their respective interpreter. Still, I'd rather have this than doing it 'manually' as a build step :)
• the line-ending of such a line must be a Unix line-ending (even on Windows), as otherwise the shell will not correctly parse it (it will include the \r in the command's filename). This in turn will probably force all other line-endings in that file to be Unix too, to prevent messages when opening the file in an editor, source-code-control messages, etc.
• on Linux, the file still won't be executable if the +x bit isn't set, so the compiler should probably do that automatically for such files.

[RyanCavanaugh]

Passing through only the first line this way seems plausible?

[poelstra]

@RyanCavanaugh yes, but that first line must end with (just) a line-feed, and the emitted file needs to have the executable bit set on *nix.

[mhegazy]

We do not modify file properties usually. would not that be something you do in your build script?

[poelstra]

@mhegazy yes, but then the script is still not executable (on non-Windows), so it won't help for the compile-on-save scenario.

[CyrusNajmabadi]

We could extend the TS language to call "#..." another form of trivia. We'd could then preserve it in the same way that we preserve copyright comments at the top of the file.

[poelstra]

That would be a useful part of the solution yes.

For reference, our use-case at work is that we have quite a few handy little utils in our codebase, to restart services, initialize databases, etc. Often these very tools are used and developed in your typical edit-compile-run cycle while working on the main library that the tool interacts with.

For that scenario, it would be great not having to run the build system over-and-over again, as it takes at least a few seconds to run (even though we try to skip compilation for unmodified files etc).
At >100k lines of TS code, it takes quite some time just to 'initialize' gulp, and at this moment, the compile pipeline (TS, browserify, uglify, copyright, sourcemaps, etc.) is deemed too complex to 'fork' the pipeline to have a separate watch-version.

Visual Studio's compile-on-save produces almost instantaneous results, and we typically don't need more than that for most of these edit-compile-run cycles.

We develop and test on Windows, but use MSysGit's Bash as commandline. We also run all tests on a Linux machine.
So, our source files are stored with CRLF, and the emitted .js file contains CRLF. Having a simple passthrough, or 'exactly preserved' trivia will thus lead to that shebang line ending in CRLF. This doesn't work when trying to run the script in (MSysGit) Bash; many projects seem to have run into this.

On Windows, having the file output just LF would fix it: MSysGit can't use an executable bit, so it appears to read the first line of a file to see if it's a shebang, and if so, effectively assumes it's an executable.

On Linux, it's the other way around: the file would already be output with LF, but it still wouldn't be executable due to the missing +x bit. It's not that weird for compilers to set an executable bit though, that's basically what every C-compiler does too ;)

Not sure how hard this would be to add to the compiler though. I'd gladly give this up for solid CommonJS module support, to be honest ;)

TL;DR I think this would be the ingredients:

• Detect the "#!..." (note the "!" too, just "#" is invalid) at the start of a file (only then), and store it as trivia
• When emitting the file and it starts with that type of trivia:
  • Force line-endings to LF (to make it work on Windows)
  • Output that trivia as the first bytes of the file (e.g. no BOM)
  • Set the executable bit (on *nix)

[Raynos]

👍

[redchair123]

There seem to be two independent questions here:

1. how should tsc handle files with a shebang line?

2. should tsc explicitly write shebang lines if requested?

To the first question, the node interpreter can handle files with leading shebang lines:

$ cat q.js
#!/usr/bin/env node
console.log("hello world");
$ chmod +x q.js
$ node q.js
hello world
$ ./q.js
hello world

To the second question, a simple compiler flag (--shebang) that adds the line would be sufficient. We already have options that control the output, like the --module option, so this isn't totally crazy

[RyanCavanaugh]

We got confused over which of two scenarios (or maybe both) people wanted here.

Someone might have a tool that compiles and runs a TypeScript file:

#!/usr/bin/env run-ts
var x: string = 'hello world';
console.log(x);

In this case, you wouldn't want #!/usr/bin/env run-ts emitted into the output JS.

Or, someone might want to compile their TS and then run the JS separately:

#!/usr/bin/env node
var x: string = 'hello world';
console.log(x);

In this case, you would want #!/usr/bin/env node emitted into the output JS.

Is one of these much more common than the other?

[tomlarkworthy]

I use typescript to write command line node programs. e.g.
https://github.com/firebase/blaze_compiler
So the latter is more common for me (I want #/usr/bin/env node emitted
into the JS file)

On Tue, May 19, 2015 at 9:49 AM, Ryan Cavanaugh [email protected]
wrote:

We got confused over which of two scenarios (or maybe both) people wanted
here.

Someone might have a tool that compiles and runs a TypeScript file:

#/usr/bin/env run-tsvar x: string = 'hello world';console.log(x);

In this case, you wouldn't want #/usr/bin/env run-ts emitted into the
output JS.

Or, someone might want to compile their TS and then run the JS separately:

#/usr/bin/env nodevar x: string = 'hello world';console.log(x);
``
In this case, you would want#/usr/bin/env node emitted into the output JS.

Is one of these much more common than the other?

—
Reply to this email directly or view it on GitHub
#2749 (comment)
.

[CyrusNajmabadi]

It seems like we could address this with a general facility to add something to the top of the file we emit. i.e. you should be able to reference some sort of file containing preamble text that you'd like at the top of the emitted file. This would be good for things like copyright headers, shebangs, etc. And it would be nice be cause it would mean that TS wouldn't have to know about things like how people on unix want to execute files.

[jdfreder]

@RyanCavanaugh I think in the first scenario you mention, the "run-ts" tool would need to strip the shebang. My gut tells me the second scenario is more common, but that may due to a personal bias.

@CyrusNajmabadi that could work, but my initial impression is that just allowing shebang lines to pass through is the all-around simpler solution. Aside from leaking Unix-isms into Windows, is there any other issues you see with this approach? I'd like to point out that node.js handles shebangs on Windows without problem and that the npm package.json expects them to be present in files specified in bin, otherwise it won't work as mentioned in this SO response.

I know, my point is node.js & npm specific, but I still think it's important.

[jdfreder]

@RyanCavanaugh yes, but that first line must end with (just) a line-feed, and the emitted file needs to have the executable bit set on *nix.

@poelstra It seems like the appropriate solution here is for you to set the executable bit on your TS file, and TSC should set the same bits on the output file. I think even though it's related to the issue I opened (this one), I think that topic belongs it's own separate issue and shouldn't be discussed in detail here.

[poelstra]

@RyanCavanaugh Like @tomlarkworthy, I would use it in your second way (run the .js, not .ts). For your first scenario, this 'feature' wouldn't be necessary at all, because run-ts could strip the shebang.

Nitpick: the line should start with #!, not just #.

[poelstra]

@jdfreder Technically, the .ts file should not have its executable bit set, because that file cannot be executed.

Given TSC's default behavior to put compiled .js files next to the .ts, this will help with tab-completion on the shell too: it will just find the .js when I would ./he<TAB> for ./helloworld.js, instead of stopping at ./helloworld..

Let's see where this goes, if needed I'll file a separate issue for the executable bit. It will already help on Windows if we have the shebang passthrough by itself, so I certainly wouldn't want to block 'your' topic on it :)

[jdfreder]

so I certainly wouldn't want to block 'your' topic on it :)

Ah thanks :) Exactly why I brought it up.

[jesseschalken]

I have a Node.js project which I converted to TypeScript but now I have to manually add #!/usr/bin/env node to the top of the .js file after every change (which probably breaks the source map). It is not easy for me to arrange to have the .js file executed as node file.js rather than just file.js, so it would be handy if tsc just let the #!... pass through.

[jesseschalken]

@basarat Yeah, I have a few things which assume that the repo contains only a single .js file which can be deployed on it's own. I can go to the effort to fix those assumptions, but it would be handy if I didn't have to, that's all.

[jesseschalken]

@RyanCavanaugh I missed why your run-ts example required that tsc didn't emit the #!... line in it's output. I assume the theoretical run-ts command would be running the file through tsc and then to node, in which case it would be fine if the #!... was kept in the output because node ignores it.

It seems to me, after reading this thread, that the original request still stands: That tsc should tolerate a #!... line at the start of an input file and maintain it in the output file.

Regarding line endings: tsc apparently has the --newLine option to specify line endings (#1693), so you can just use that on Windows.

If you need the file to be executable, then you can chmod +x the .js file after the first compile and tsc will apparently keep it after subsequent compiles (according to my testing), and if you commit the .js file Git will also remember that it's executable.

[basarat]

@mhegazy I can implement this quite easily. Basically do it the same we parse out reference comments and store it in SourceFile, and then write it out on emit as is.

This is very similar to how babel does it as well (calls parseShebang at the start, and if the first line matches the regex stores it in a shebang property on what is effectively its sourceFile and then writes it out on emit). Would you be okay with a PR?

[DanielRosenwasser]

@basarat I think the problem is that there's no consensus yet on what should be done with a shebang.

@jesseschalken If a shebang is meant to be preserved, it implies that the .ts file is not run - the resultant .js is. But clearly there is a use case for other tools to run the .ts file directly with no resultant .js file.

One could imagine us omitting shebangs on output, but also allowing a prelude as @CyrusNajmabadi suggested.

[basarat]

I think the problem is that there's no consensus yet on what should be done with a shebang

Which is why I brought up babel to present a precedence. It just preserves it 🌹

[mhegazy]

@basarat we should do that. let me get back to you tomorrow, i will bring it up in the design meeting.

[jesseschalken]

@DanielRosenwasser "But clearly there is a use case for other tools to run the .ts file directly with no resultant .js file."

I must be missing something. The only two ways some tool could "run" a .ts file would be:

• Convert the TS into JS using tsc and pipe that into a JS interpreter such as node. node ignores a #! line in it's input, so it's fine if tsc kept it in its output. If the tool is using some other JS interpreter that doesn't support a #! line, then the tool can remove it.
• Somehow run the TS without converting it to JS, in which case it won't be using tsc in the first place.

[mhegazy]

@basarat feel free to send a PR for this

[CyrusNajmabadi]

@basarat

This is very similar to how babel does it as well (calls parseShebang at the start, and if the first line matches the regex stores it in a shebang property on what is effectively its sourceFile and then writes it out on emit). Would you be okay with a PR?

Sounds good. However, i would prefer you parse it out as trivia instead of adding it as a special field on sourceFile. As an example of where we have specialized trivia parsing, see the code i added so we can detect and handle git merge markers (i.e. "<<<<<< HEAD"). Then we can even do nice stuff like classify this guy properly in the editor and not barf on it while formatting. yadda yadda yadda.

If you wanted to send me the shebang syntax (it's just #! <stuff to end of line> right?), then i'd be happy to whip this up myself.

[basarat]

Then we can even do nice stuff like classify this guy properly in the editor and not barf on it while formatting

👍 🌹 SyntaxKind.ShebangTrivia

[basarat]

If you wanted to send me the shebang syntax (it's just #! right?), then i'd be happy to whip this up myself.

@CyrusNajmabadi Yup. I have some work on the scanner based on your suggestion that might be helpful https://github.com/Microsoft/TypeScript/compare/master...basarat:feat/shebang?expand=1

Looking forward to your work 🌹

[basarat]

I just PRed it

[jdfreder]

Awesome, thanks for moving forward with this everyone!

[mhegazy]

thanks @basarat

[basarat]

🌹

[tomlarkworthy]

[image: Office dance]

On Tue, Aug 4, 2015 at 2:42 PM, Basarat Ali Syed [email protected]
wrote:

🌹

—
Reply to this email directly or view it on GitHub
#2749 (comment)
.

[ghost]

Thank you for fixing this! :)

[woodie]

I came up with a simple solution, described here.
https://medium.com/@JohnWoodell/time-for-typescript-e87b78a796d0#.a1694wfih

[DanielRosenwasser]

@woodie nice. As a heads up, check out our wiki's editor support page to get some vim plugins and more updated sytnax files.