Skip to content

Commit 29b1106

Browse files
authored
[node] Switch to setImmediate to avoid starving the Node.js event loop (#19610)
### Description <!-- Describe your changes. --> Switch to setImmediate to avoid starving the Node.js event loop There should really be a true async version though, running computationally intensive things on the event loop will stop everything else from happening while it is running, e.g. a web server from answering requests. This can be done by wrapping `RunAsync` behind a [`napi::Promise`](https://github.com/nodejs/node-addon-api/blob/main/doc/promises.md) to run on the onnxruntime thread pool or [`AsyncWorker`]( https://github.com/nodejs/node-addon-api/blob/main/doc/async_worker.md) for the Node.js/libuv thread pool. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Without this, if you run inference in a tight loop, without anything else in between that is async/deferred, `process.nextTick` will lead to starving the event loop and not letting anything else run, `setImmediate` at least lets the event loop spin between calls to `run`. See https://dev.to/ynmanware/setimmediate-settimeout-and-process-nexttick-3mfd Contributed on behalf of [Swimm](https://swimm.io/)
1 parent 4ab4976 commit 29b1106

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

js/node/lib/backend.ts

+2-2
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ class OnnxruntimeSessionHandler implements InferenceSessionHandler {
3636
async run(feeds: SessionHandler.FeedsType, fetches: SessionHandler.FetchesType, options: InferenceSession.RunOptions):
3737
Promise<SessionHandler.ReturnType> {
3838
return new Promise((resolve, reject) => {
39-
process.nextTick(() => {
39+
setImmediate(() => {
4040
try {
4141
resolve(this.#inferenceSession.run(feeds, fetches, options));
4242
} catch (e) {
@@ -56,7 +56,7 @@ class OnnxruntimeBackend implements Backend {
5656
async createInferenceSessionHandler(pathOrBuffer: string|Uint8Array, options?: InferenceSession.SessionOptions):
5757
Promise<InferenceSessionHandler> {
5858
return new Promise((resolve, reject) => {
59-
process.nextTick(() => {
59+
setImmediate(() => {
6060
try {
6161
resolve(new OnnxruntimeSessionHandler(pathOrBuffer, options || {}));
6262
} catch (e) {

0 commit comments

Comments
 (0)