Allow pthreads on Node.js without a pthread pool

[RReverser]

Node.js worker_threads are different than browser Workers in that they start spawning synchronously without waiting for an event loop tick.

If we can also avoid waiting for the "loaded" event from the worker before sending pthread tasks to it, then we can spawn new pthreads "synchronously" (prepare everything within the same event loop tick), and block the current thread, making the behaviour on Node.js a lot closer to native and avoid the need for a pthread pool even without Asyncify or extra helper workers.

That's what I did in this implementation. Instead of waiting for the worker to tell us it's loaded and ready, I'm sending all commands immediately to the worker. The worker accepts the first "load" message, starts initializing the runtime, and meanwhile queues up any further messages such as "run". Once the runtime is ready, it processes the queue.

I could limit those changes only to Node.js, but it's easier to do both together, as it allows to avoid a custom worker.runPthread callback, and should be in theory even a bit faster in browsers too (by avoiding the "loaded" roundtrip before running a pthread).

[RReverser]

Note: this depends on #18267, because it changes the time of when the "run" command is sent to a Worker. That one's merged now.

[kripken]

Maybe someone else can review this more easily, but for me at least I don't recall all the async startup logic, so I could benefit from an overview of the old way and the new way proposed here.

[RReverser]

Maybe someone else can review this more easily, but for me at least I don't recall all the async startup logic, so I could benefit from an overview of the old way and the new way proposed here.

Let me try to do a TL;DR summary by using a pthread_create as an example and assuming we don't have a pthread pool or it's exhausted.

Before:

1. Main thread creates a Worker via new Worker and subscribes to its messages.
2. Main thread sends a message "load" with the Wasm module and some other bits to the Worker.
3. Worker receives the "load" message with the Wasm module, loads relevant JS files, and asynchronously initializes the runtime.
4. Worker initialization is done, it sends a message "loaded" to the main thread.
5. Main thread receives message "loaded".
6. Main thread sends a message "run" to the Worker with a pointer to pthread callback and other relevant info.
7. Worker receives the message "run" and executes the pthread.

After:

1. Main thread creates a Worker via new Worker and subscribes to its messages.
2. Main thread sends a message "load" with the Wasm module and some other bits to the Worker.
3. Main thread sends a message "run" to the Worker with a pointer to pthread callback and other relevant info.
4. Worker receives the "load" message with the Wasm module, loads relevant JS files, and asynchronously initializes the runtime.
5. Worker stores all the other incoming messages into a queue (in this example it's just a message "run").
6. Worker initialization is done, it sends a message "loaded" to the main thread.
7. Worker executes all the queued up messages (in this example just a message "run", so it executes the pthread).

What this reordering means on practice in Node.js is that before this change the main thread couldn't block synchronously right after pthread_create because it then didn't ever have a chance to run steps 5-7, because, with the event loop blocked, it wouldn't even receive the message "loaded" from the Worker.

After this change, the main thread doesn't wait for that "loaded" message anymore and instead sends the "run" command immediately in the same event loop tick, putting the responsibility onto the Worker to only execute the code after it's fully loaded. This means main thread can safely block right after the pthread_create call, even though the Worker is not ready yet.

Hope this helps.

[kripken]

@RReverser In the "after" list, steps 2 and 3 are both "Main thread sends a message" - could it not send a single message for both?

[RReverser]

@RReverser In the "after" list, steps 2 and 3 are both "Main thread sends a message" - could it not send a single message for both?

They're not always sent together. In this particular example - pthread_create when a threadpool is not ready - yes, it will send 2 messages. However, in other cases, e.g. when preparing a threadpool, there will be only "load" message, and when doing pthread_create with a ready pool, there will be only "run".

It's possible to add 3rd kind of message for combined "loadAndRun", but it would complicate code further and won't add any perf benefits because those messages, when they happen to be sent both at once, are already sent in the same event loop tick.

[RReverser]

Rebased & added a changelog line. Are there any blocking changes here left?

[sbc100]

So IIRC that pool pre-allocation still happens under node, even though its not really necessary?

I do like being able to test stuff under node, so I guess it is still useful, but should we consider having the re-allocated pool be something that only ever happens in the browser?

[RReverser]

So IIRC that pool pre-allocation still happens under node, even though its not really necessary?

If user specifies PTHREAD_POOL_SIZE=N, then yes. But if not specified or if the pool is exceeded, then it won't throw an error anymore, but will spawn synchronously anyway.

but should we consider having the re-allocated pool be something that only ever happens in the browser?

I think it makes sense to still respect user choice here; if they explicitly specify PTHREAD_POOL_SIZE, they might want the faster startup for pthreads.

Once we add some way to avoid pthread pool in browsers too, we can document that PTHREAD_POOL_SIZE is no longer necessary if your only goal is to avoid deadlocks, and gradually migrate users away from it.

I don't want to completely ignore the option just for one target as part of this PR though.

[kripken]

They're not always sent together. In this particular example - pthread_create when a threadpool is not ready - yes, it will send 2 messages. However, in other cases, e.g. when preparing a threadpool, there will be only "load" message, and when doing pthread_create with a ready pool, there will be only "run".

I see, thanks, that's what I was missing.

[RReverser]

I see, thanks, that's what I was missing.

Glad it clears it up, sorry for the confusion. I used that particular scenario to show differences in workflows because it's the scenario that would deadlock before this change but won't now; other mentioned scenarios are less interesting because they worked anyway.