Skip to content

stream output's type #213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
callanwu opened this issue Dec 4, 2023 · 1 comment · Fixed by #214
Closed

stream output's type #213

callanwu opened this issue Dec 4, 2023 · 1 comment · Fixed by #214

Comments

@callanwu
Copy link

callanwu commented Dec 4, 2023

When i use the provided steam output's example code(https://github.com/replicate/replicate-python#run-a-model-and-stream-its-output), it output nothing but output blank lines.
Is this related to the type of event?
May be the code should be

import replicate

# https://replicate.com/meta/llama-2-70b-chat
model_version = "meta/llama-2-70b-chat:02e509c789964a7ea8736978a43525956ef40397be9033abf9fd2badfe68c9e3"

tokens = []
for event in replicate.stream(
    model_version,
    input={
        "prompt": "Please write a haiku about llamas.",
    },
):
    print(event.data)
    tokens.append(str(event.data))

print(" ".join(tokens))

But when I use the above code, it always output a {} at the end.
Could you provide a correct code example?

@mattt mattt closed this as completed in #214 Dec 4, 2023
mattt added a commit that referenced this issue Dec 4, 2023
Resolves #213 

This PR fixes a few bugs in the streaming implementation added with
#204.

---------

Signed-off-by: Mattt Zmuda <[email protected]>
@mattt
Copy link
Contributor

mattt commented Dec 4, 2023

Hi @callanwu. Thanks for reporting, and sorry for any convenience. I found some bugs in the stream method that I fixed in #214. Those are now available in 0.21.1. Please let me know if you have any more issues with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants