Closed
Description
Context
#1595 adds support for running models without a gateway, in which case the model is served by dstack-server and the model URL consists only of a dstack-server path. Frontend changes are required to correctly display such URLs on the Models page (/models
) and use them with the OpenAI client in the model chat.
Current behavior
- The table on the Models page has a column named "Gateway".
- The
service.model.base_url
field returned by/api/runs/get
or/api/runs/list
always contains a full URL that is then displayed in the "Gateway" column and passed to the OpenAI client as is.
New expected behavior
- The column should be renamed "URL".
- The
service.model.base_url
field may contain either a full URL or only a path.- If it's a full URL, it should be displayed in the "URL" column and passed to the OpenAI client as is.
- If it's a path, the URL displayed in the "URL" column and passed to the OpenAI client should be constructed by adding that path to the dstack-server base URL. 1
Examples
dstack-server base URL | The service.model.base_url field |
Resulting model URL |
---|---|---|
any | https://gateway.admin.sky.dstack.ai |
https://gateway.admin.sky.dstack.ai |
any | http://gateway.mycompany.example |
http://gateway.mycompany.example |
https://sky.dstack.ai |
/proxy/models/admin/ |
https://sky.dstack.ai/proxy/models/admin/ |
http://localhost:3000 |
/proxy/models/admin/ |
http://localhost:3000/proxy/models/admin/ |
Footnotes
-
The dstack-server base URL here means the scheme, hostname, and optional port. The
/api
prefix is not part of the base URL. ↩