Skip to content

Commit ce60541

Browse files
authored
[transformers] update text gen server example (#1440)
1 parent 11c9225 commit ce60541

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

src/deepsparse/transformers/README.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,7 @@ Spinning up:
163163
```bash
164164
deepsparse.server \
165165
task text-generation \
166-
--model_path zoo:llama2-7b-open_platypus_orca_llama2_pretrain-pruned50_quantized
166+
--model_path zoo:opt-1.3b-opt_pretrain-pruned50_quantW8A8
167167
```
168168

169169
Making a request:
@@ -172,12 +172,12 @@ import requests
172172

173173
url = "http://localhost:5543/v2/models/text_generation/infer" # Server's port default to 5543
174174

175-
obj = {"prompt": "Who is the president of the United States?"}
175+
obj = {"prompt": "Large language models are"}
176176

177177
response = requests.post(url, json=obj)
178178
print(response.json()["generations"][0]["text"])
179179

180-
>> 'The president of the United States is the head of the executive branch of government...'
180+
>> ' often used to model the language of a large number of users...'
181181
```
182182

183183
### Sentiment Analysis

0 commit comments

Comments
 (0)