|
| 1 | +# API usage - SessionOptions |
| 2 | + |
| 3 | +## Summary |
| 4 | + |
| 5 | +This example is a demonstration of how to configure an `InferenceSession` instance using `SessionOptions`. |
| 6 | + |
| 7 | +A `SessionOptions` is an object with properties to instruct the creation of an `InferenceSession` instance. See [type declaration](https://github.com/microsoft/onnxruntime/blob/master/js/common/lib/inference-session.ts) for schema definition. `SessionOptions` is passed to `InferenceSession.create()` as the last parameter: |
| 8 | + |
| 9 | +```js |
| 10 | +const mySession = await InferenceSession.create(..., mySessionOptions); |
| 11 | +``` |
| 12 | + |
| 13 | +### Execution providers |
| 14 | + |
| 15 | +An [execution provider](https://onnxruntime.ai/docs/reference/execution-providers/) (EP) defines how operators get resolved to specific kernel implementation. Following is a table of supported EP in different environments: |
| 16 | + |
| 17 | +| EP name | Hardware | available in | |
| 18 | +| ------- | ----------------- | --------------------------------- | |
| 19 | +| `cpu` | CPU (default CPU) | onnxruntime-node | |
| 20 | +| `cuda` | GPU (NVIDIA CUDA) | onnxruntime-node | |
| 21 | +| `dml` | GPU (Direct ML) | onnxruntime-node (Windows) | |
| 22 | +| `wasm` | CPU (WebAssembly) | onnxruntime-web, onnxruntime-node | |
| 23 | +| `webgl` | GPU (WebGL) | onnxruntime-web | |
| 24 | + |
| 25 | +Execution provider is specified by `sessionOptions.executionProviders`. Multiple EPs can be specified and the first available one will be used. |
| 26 | + |
| 27 | +Following are some example code snippets: |
| 28 | + |
| 29 | +```js |
| 30 | +// [Node.js binding example] Use CPU EP. |
| 31 | +const sessionOption = { executionProviders: ['cpu'] }; |
| 32 | +``` |
| 33 | + |
| 34 | +```js |
| 35 | +// [Node.js binding example] Use CUDA EP. |
| 36 | +const sessionOption = { executionProviders: ['cuda'] }; |
| 37 | +``` |
| 38 | + |
| 39 | +```js |
| 40 | +// [Node.js binding example] Use CUDA EP, specifying device ID. |
| 41 | +const sessionOption = { |
| 42 | + executionProviders: [ |
| 43 | + { |
| 44 | + name: 'cuda', |
| 45 | + deviceId: 0 |
| 46 | + } |
| 47 | + ] |
| 48 | +}; |
| 49 | +``` |
| 50 | + |
| 51 | +```js |
| 52 | +// [Node.js binding example] Try multiple EPs using an execution provider list. |
| 53 | +// The first successfully initialized one will be used. Use CUDA EP if it is available, otherwise fallback to CPU EP. |
| 54 | +const sessionOption = { executionProviders: ['cuda', 'cpu'] }; |
| 55 | +``` |
| 56 | + |
| 57 | +```js |
| 58 | +// [ONNX Runtime Web example] Use WebAssembly (CPU) EP. |
| 59 | +const sessionOption = { executionProviders: ['wasm'] }; |
| 60 | +``` |
| 61 | + |
| 62 | +```js |
| 63 | +// [ONNX Runtime Web example] Use WebGL EP. |
| 64 | +const sessionOption = { executionProviders: ['webgl'] }; |
| 65 | +``` |
| 66 | + |
| 67 | +### other common options |
| 68 | + |
| 69 | +There are also some other options available for all EPs. |
| 70 | + |
| 71 | +Following are some example code snippets: |
| 72 | + |
| 73 | +```js |
| 74 | +// [Node.js binding example] Use CPU EP with single-thread and enable verbose logging. |
| 75 | +const sessionOption = { |
| 76 | + executionProviders: ['cpu'], |
| 77 | + interOpNumThreads: 1, |
| 78 | + intraOpNumThreads: 1, |
| 79 | + logSeverityLevel: 0 |
| 80 | +}; |
| 81 | +``` |
| 82 | + |
| 83 | +```js |
| 84 | +// [ONNX Runtime Web example] Use WebAssembly EP and enable profiling. |
| 85 | +const sessionOptions = { |
| 86 | + executionProviders: ['wasm'], |
| 87 | + enableProfiling: true |
| 88 | +}; |
| 89 | +``` |
| 90 | + |
| 91 | +See also [`SessionOptions` interface](https://onnxruntime.ai/docs/api/js/interfaces/InferenceSession.SessionOptions.html) in API reference document. |
| 92 | + |
| 93 | +### SessionOptions vs. ort.env |
| 94 | + |
| 95 | +Both `SessionOptions` and `ort.env` allow to specify configurations for inferencing behaviors. The biggest difference of them is: `SessionOptions` is set for one inference session instance, while `ort.env` is set global. |
| 96 | + |
| 97 | +See also [API usage - `ort.env` flags](../api-usage_ort-env-flags) for an example of using `ort.env`. |
| 98 | + |
| 99 | +## Usage |
| 100 | + |
| 101 | +The code snippets demonstrated above cannot run standalone. Put the code into one of the "Quick Start" examples and try it out. |
0 commit comments