Async calls do synchronous sleeping while waiting for predictions.

[LucasSwitz]

It looks like the replicate client is synchronously waiting for predictions to finish during async methods.

replicate_client.async_run eventually calls this function that waits on the prediction: https://github.com/replicate/replicate-python/blob/main/replicate/run.py

prediction.wait() looks to be a synchronously spinning function: https://github.com/replicate/replicate-python/blob/main/replicate/prediction.py#L122

Am I understanding this correctly? Can we add a prediction.async_wait() that uses asyncio.wait() instead of time.sleep().

[rohan-mehta]

#225 fixes this partly, wanted to note that prediction.output_iterator() still does a blocking sleep: https://github.com/replicate/replicate-python/blob/a3784c43503fa4f1193b7b5a1c84bca662088c6d/replicate/prediction.py#L200

That is called from _make_output_iterator in async_run.

[mattt]

Hi @LucasSwitz and @rohan-mehta. Thanks for reporting this. I just opened #230, which should address this. Please take a look and let me know if that works as expected.

[mattt]

This is now available in version 0.23.1.