You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Fixes#287
Given the following code:
```python
import replicate
def go():
for event in replicate.stream(
"meta/meta-llama-3-8b-instruct",
input={
"top_p": 0.9,
"prompt": "Hi! Help me please:)",
"max_tokens": 512,
"min_tokens": 0,
"temperature": 0.01,
"stop_sequences": "<|end_of_text|>",
"prompt_template": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\n{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n",
"presence_penalty": 0,
"frequency_penalty": 1,
},
):
print(str(event), end="")
print()
go()
print("---------------------------------")
go()
print("---------------------------------")
go()
print("---------------------------------")
```
The latest release repeats the same token once for each previous
invocation.
<details>
```
Hi there! I'd be happy to help you with whatever you need. What's on your mind?assistant
I'm glad you asked! I'm here to help you with any questions or problems you might have. Whether it's something big or something small, I'm here to help.
So, what's on your mind? Do you have a specific question or problem you'd like to talk about?assistant
I'm all ears! Take your time, and feel free to share as much or as little as you'd like.
Remember, everything we discuss is confidential and just between us. So,
---------------------------------
Hi
Hi there! I'd be happy to help you with whatever you need. What's on your mind?assistant
I'm glad you asked! I'm here to help you with any questions or problems you might have. Whether it's something big or something small, I'm here to help.
So, what's on your mind? Do you have a specific question or problem you'd like to talk about?assistant
I'm all ears! Take your time, and feel free to share as much or as little as you'd like.
Remember, everything we discuss is confidential and just between us. So,
---------------------------------
Hi
Hi
Hi there! I'd be happy to help you with whatever you need. What's on your mind?assistant
I'm glad you asked! I'm here to help you with any questions or problems you might have. Whether it's something big or something small, I'm here to help.
So, what's on your mind? Do you have a specific question or problem you'd like to talk about?assistant
I'm all ears! Take your time, and feel free to share as much or as little as you'd like.
Remember, everything we discuss is confidential and just between us. So,
---------------------------------
```
</details>
This is caused by incorrect initialization of the stream `Decoder`
object. After applying this change, the code produces the correct
behavior:
<details>
```
Hi there! I'd be happy to help you with whatever you need. What's on your mind?assistant
I'm glad you asked! I'm here to help you with any questions or problems you might have. Whether it's something big or something small, I'm here to help.
So, what's on your mind? Do you have a specific question or problem you'd like to talk about?assistant
I'm all ears! Take your time, and feel free to share as much or as little as you'd like.
Remember, everything we discuss is confidential and just between us. So,
---------------------------------
Hi there! I'd be happy to help you with whatever you need. What's on your mind?assistant
I'm glad you asked! I'm here to help you with any questions or problems you might have. Whether it's something big or something small, I'm here to help.
So, what's on your mind? Do you have a specific question or problem you'd like to talk about?assistant
I'm all ears! Take your time, and feel free to share as much or as little as you'd like.
Remember, everything we discuss is confidential and just between us. So,
---------------------------------
Hi there! I'd be happy to help you with whatever you need. What's on your mind?assistant
I'm glad you asked! I'm here to help you with any questions or problems you might have. Whether it's something big or something small, I'm here to help.
So, what's on your mind? Do you have a specific question or problem you'd like to talk about?assistant
I'm all ears! Take your time, and feel free to share as much or as little as you'd like.
Remember, everything we discuss is confidential and just between us. So,
---------------------------------
```
Signed-off-by: Mattt Zmuda <[email protected]>
0 commit comments