Question: Interactive Development Workflow

[Michael-F-Ellis]

In day-to-day Python development, I almost always keep an iPython session active so I can easily paste a few lines for a quick test of functionality or to look up a docstring from an infrequently used module.

Chrome DevTools serves similarly for web development. I use the console to test JS functions and expressions and to explore library functions.

Since Transcrypt is a compiler, there's inevitably some loss of immediacy. Although I can still test pure python expressions in iPython and pure JS expressions in DevTools, I'm wondering about recommended practices for developing with Transcrypt. The source maps work well for debugging -- that's a plus and not much of a concern. The more urgent question, I think, is how to efficiently explore unfamiliar JS libraries and figure out how to best use them from Python, e.g. what types of arguments to pass and and how to access returned objects.

I realize this isn't something to which there's a one-size-fits-all answer so I see this issue as perhaps the start of a discussion thread. I'd be particularly interested to hear about how the examples that use React, Plotly, etc were developed.

Cheers,
Mike

[JdeH]

Indeed there's no general 'best way', so this should indeed be a 'growing' thread.

Even in my own way of developing with Transcrypt there are some variations.

• The Plotly example was converted directly from a JavaScript Plotly example. Once it worked, some things were added and altered, such as the list comprehensions. The main difference encountered when converting from JS to TS is that Python/TS, apart from lambda's, doesn't know anonymous functions. So 'normal' functions or bound methods are used instead. Another tiny difference is that string literals used as dictionary keys should be enclosed in quotes, whereas JS doesn't enforce this.

• The pong example was converted from an existing Python program almost unmodified. Only was the use of the Pyglet library replaced by use of the Fabric.js library (which was overkill, working directly on the HTML5 canvas would have been simpler and much less bloated).

• The turtle graphics client/server example was programmed directly in Transcrypt, which by that time felt quite familiar to me. In fact if I have to program JS now I constantly have to look up things, whereas Transcrypt is just, well, Python. Note the way Ajax is done in TS / JQuery. Not much difference with doing it in JS / JQuery. Recently there was a question about this on SO:
  http://stackoverflow.com/questions/43494118/transcrypt-how-to-send-an-ajax-post-request-to-start-a-server-side-job-using-py

• Less trivial algorithms, like the ones in Numscrypt, were tested back to back with CPython.

• In general I have a HTML 'testbed' ready. When developing locally, I just compile with the right switches (and for certainty, the -b switch), and with sourcemaps. Then I click the HTML file to view the result in the browser. If there's a lot of computation or logic going on, I make a Transcrypt terminal application first and test it using a (very limited, simplified) terminal window as in the terminal_demo. Note that this approach is effective but inelegant, since browsers are by nature event driven. Having continuations as in Skulpt I consider overkill for these bare bones tests

• If the DOM plays a large role in an application I tend to work in small increments, in most cases not using a server but just loading the HTML files in which the references to the generated JS are made and then hitting 'refresh' on the browser after each modification/compilation.

• In more complicated cases I upload repeatedly to a server. I use e.g. a free account at www.alwaysdata.com for those cases. It allows me to run Python 3.6 on the server as well, so I can work in one language on client and server.

• I more often use a mental model than a debugger, but if console output indicate things going wrong, the sourcemaps often help.

• When testing Trancrypt code, I often add print statements in many places. These perform output to the console and allow me to follow the control flow a bit.

• The React example, if I remember well, was contributed by @pierrejean-coudert . Perhaps he can shed some light upon how he constructed it.

• An incremental way to convert from JS (used as examples in the docs of JS libraries) to TS, is to make liberal use of __pragma__ ('js', '{}', <js code>) first and then gradually replace those pragma's by Transcrypt code. This is especially handy in the beginning. Later on just thinking Python and skipping the JS mental step is much easier.

• To gain some speed during development I compile without minification, since that costs relatively much time. The compiler itself is quite fast.

• When working on a larger multi-module project I use type annotations on the module boundaries and the -ds switch (static type checking with mypy) switch. It's some up front investment of effort, but it catches a lot of errors early.

• Yet another small trick for non-GUI/non-DOM code is to first compile with transcrypt -r -da. It will invoke the CPython interpreter, but with the right library paths for Transcrypt. This catches runtime errors directly in Python, rather than in the compiled JS.

• While using minification is relatively slow, the Google Closure compiler/minifier catches a lot of quirks in the generated JS code, such as unreachable code. Only the lowest compression level is used. All other levels generate subtly deviating semantics, at least the last time I tried, about a year ago.

Well, all in all not very ground breaking, but I hope it gives you an idea.

[Michael-F-Ellis]

Thanks for the very detailed reply! It's helpful indeed.

So 'normal' functions or bound methods are used instead

That makes sense. I've always thought JS programmers tend to overdo it with anonymous functions in the sense that a little reduction in code size or a small gain in speed may not be worth the loss of readability.

use of the Fabric.js library (which was overkill, working directly on the HTML5 canvas would have been simpler and much less bloated).

I suspect there may be a lot of opportunity for working more directly with the DOM as Python makes it, to my mind, a little easier to wrap the DOM calls into functions and classes.

In fact if I have to program JS now I constantly have to look up things, whereas Transcrypt is just, well, Python.

That's encouraging. I'm hoping the same happens with me after I've written more TS.

Less trivial algorithms, like the ones in Numscrypt, were tested back to back with CPython.

Make perfect sense for anything that doesn't use a JS library or need to be running in a browser.

In general I have a HTML 'testbed' ready.

The NearlyPurePythonWebAppDemo is my way of doing that. By running it locally with Bottle's reloader enabled, server-side changes update automatically. For the client side, I can change/save/recompile the Transcrypt files and then do a page reload without restarting the server.

a free account at www.alwaysdata.com for those cases. It allows me to run Python 3.6 on the server as well, so I can work in one language on client and server.

I'm using a $5/month account at pythonanywhere.com. I'm pretty happy with them so far.

print statements in many places

Sometimes the old ways are best :-)

to make liberal use of pragma ('js', '{}', ) first and then gradually replace

That's a really handy feature.

compile without minification

Definitely a timesaver.

type annotations on the module boundaries and the -ds switch (static type checking with mypy)

Ok. I'm going to have read that section of your docs a few more times and figure out how to use that.

While using minification is relatively slow, the Google Closure compiler/minifier catches a lot of quirks in the generated JS code, such as unreachable code. Only the lowest compression level is used. All other levels generate subtly deviating semantics, at least the last time I tried, about a year ago.

I'll give that a try. And thanks for the heads-up about the deviations.

[fzzylogic]

For client side auto-compile, one can use use something like entr, as suggested by AXGKI a while back. It's linux based, but if you're on Win 10, "Windows subsystem for Linux" has a nice Bash shell (not VM based, but using a system that translates linux kernel calls to windows kernel calls, so it's fast and does not use too many resources). You may wish to use the version distributed with the 'creators update', as the previous version had some issues, though it did what i needed for Transcrypting - i was also able to minify after installing java using apt-get, despite that they said java didn't work.

Using entr in combination with tmux, one can make something quite useful. For example:

tmux \
       new-session -c './' 'while true; do ls ./src/*.py | entr -c -d transcrypt -b -m mainscript.py; done' \; \
       split-window -c '../' 'echo "Server listening at localhost:5000"; busybox httpd -f -p 5000' \; \
       split-window -c './src' '/bin/bash' \; \
       select-layout tiled

This opens three views within the bash terminal, one for auto compiling whenever ./src/*.py is changed, one with a web server and one with a normal console for doing things. From AXGKI's example above, one could also modify the first session to allow auto-reloading of the browser window.

Edit: Just realised i haven't tested entr or tmux on Windows yet, but the above works in bash on linux.

[Michael-F-Ellis]

Hey, fzzylogic, thanks for the pointer to enter. I installed and played with it in OS X. Seems to work nicely. I suspect I'll find a lot of uses for it as time goes on.

In the meantime, I've committed some refinements to my demo skeleton that are making the development process very convenient. It now rebuilds and reloads both the server and the client pages whenever I change any of the source files. Full documentation here.

The above plus Chrome DevTools are pretty close to all I could ask for at this time. Now if I could only do something about my error-prone brain ...

[fzzylogic]

Thanks for the update, it's looking good! I'm also glad about the removal of jQuery. Don't have any issues with it, but prefer not to use it unless i have to.

[JdeH]

This thread seems to have come to a standstill.
Feel free to reopen if needed!