Skip to content

Commit 389eaa5

Browse files
thundergolfergongy
authored andcommitted
fix: don't loop forever if TGI server fails (#456)
1 parent 4915585 commit 389eaa5

File tree

1 file changed

+7
-0
lines changed

1 file changed

+7
-0
lines changed

06_gpu_and_ml/text_generation_inference.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,13 @@ def webserver_ready():
122122
socket.create_connection(("127.0.0.1", 8000), timeout=1).close()
123123
return True
124124
except (socket.timeout, ConnectionRefusedError):
125+
# Check if launcher webserving process has exited.
126+
# If so, a connection can never be made.
127+
retcode = self.launcher.poll()
128+
if retcode is not None:
129+
raise RuntimeError(
130+
f"launcher exited unexpectedly with code {retcode}"
131+
)
125132
return False
126133

127134
while not webserver_ready():

0 commit comments

Comments
 (0)