You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/sentence_transformer/usage/efficiency.rst
+11-1Lines changed: 11 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -99,7 +99,11 @@ To convert a model to ONNX format, you can use the following code:
99
99
sentences = ["This is an example sentence", "Each sentence is converted"]
100
100
embeddings = model.encode(sentences)
101
101
102
-
If the model path or repository already contains a model in ONNX format, Sentence Transformers will automatically use it. Otherwise, it will convert the model to ONNX the format.
102
+
If the model path or repository already contains a model in ONNX format, Sentence Transformers will automatically use it. Otherwise, it will convert the model to the ONNX format.
103
+
104
+
.. note::
105
+
106
+
If you wish to use the ONNX model outside of Sentence Transformers, you'll need to perform pooling and/or normalization yourself. The ONNX export only converts the Transformer component, which outputs token embeddings, not sentence embeddings. To get sentence embeddings, you'll need to apply the appropriate pooling strategy (like mean pooling) and any normalization that the original model uses.
103
107
104
108
All keyword arguments passed via ``model_kwargs`` will be passed on to :meth:`ORTModel.from_pretrained <optimum.onnxruntime.ORTModel.from_pretrained>`. Some notable arguments include:
105
109
@@ -291,6 +295,12 @@ To convert a model to OpenVINO format, you can use the following code:
291
295
sentences = ["This is an example sentence", "Each sentence is converted"]
292
296
embeddings = model.encode(sentences)
293
297
298
+
If the model path or repository already contains a model in OpenVINO format, Sentence Transformers will automatically use it. Otherwise, it will convert the model to the OpenVINO format.
299
+
300
+
.. note::
301
+
302
+
If you wish to use the OpenVINO model outside of Sentence Transformers, you'll need to perform pooling and/or normalization yourself. The OpenVINO export only converts the Transformer component, which outputs token embeddings, not sentence embeddings. To get sentence embeddings, you'll need to apply the appropriate pooling strategy (like mean pooling) and any normalization that the original model uses.
303
+
294
304
.. raw:: html
295
305
296
306
All keyword arguments passed via <code>model_kwargs</code> will be passed on to <ahref="https://huggingface.co/docs/optimum/intel/openvino/reference#optimum.intel.openvino.modeling_base.OVBaseModel.from_pretrained"><codestyle="color: #404040; font-weight: 700;">OVBaseModel.from_pretrained()</code></a>. Some notable arguments include:
0 commit comments