File tree 1 file changed +3
-3
lines changed
src/deepsparse/transformers
1 file changed +3
-3
lines changed Original file line number Diff line number Diff line change @@ -163,7 +163,7 @@ Spinning up:
163
163
``` bash
164
164
deepsparse.server \
165
165
task text-generation \
166
- --model_path zoo:llama2-7b-open_platypus_orca_llama2_pretrain-pruned50_quantized
166
+ --model_path zoo:opt-1.3b-opt_pretrain-pruned50_quantW8A8
167
167
```
168
168
169
169
Making a request:
@@ -172,12 +172,12 @@ import requests
172
172
173
173
url = " http://localhost:5543/v2/models/text_generation/infer" # Server's port default to 5543
174
174
175
- obj = {" prompt" : " Who is the president of the United States? " }
175
+ obj = {" prompt" : " Large language models are " }
176
176
177
177
response = requests.post(url, json = obj)
178
178
print (response.json()[" generations" ][0 ][" text" ])
179
179
180
- >> ' The president of the United States is the head of the executive branch of government ...'
180
+ >> ' often used to model the language of a large number of users ...'
181
181
```
182
182
183
183
### Sentiment Analysis
You can’t perform that action at this time.
0 commit comments