-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Revert name change to make pool name more descriptive. #516
Conversation
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: kfswain The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
✅ Deploy Preview for gateway-api-inference-extension ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
I sent #515 to also change the name of the ext_proc.yaml file to inferencepool.yaml and to align the naming of resources that gets created for the pool (the deployment and service). The two PRs will conflict now :) |
@ahg-g @kfswain the name is now descriptive for the gpu deployment, but using the same name for the cpu deployment may be very confusing, as the cpu deployment do NOT use the llama2 model. my original intention was to have a label that can be used for any of the deployments and describe a general pool. I think this change should be reverted and maybe if we want to describe the deployment with the concrete model we can put additional label with |
I agree, we can call it pool-1, we will have #416 merge soon and I think a generic name makes sense. |
With the addition of the CPU deployment path, we recently changed the name to
my-pool
which is so generic as to not mean much of anything.Reverting this change to give some context clues as to what an InferencePool is intended to house.
vllm-llama2-7b-pool
is fairly descriptive in what the pool contains.We will soon have helm charts: #416 so this name can be more configurable. But for the getting started guide in the short term I would rather make an opinionated decision on the naming.