Skip to content

[V1] Why there is no swap queue in V1 version Scheduler? #11082

Closed Answered by comaniac
Ghjk94522 asked this question in Q&A
Discussion options

You must be logged in to vote

Swapping in v1 is removed intentionally. Instead, we expect when recomputing the preempted requests, most prompt tokens could be bypassed due to prefix caching.

Replies: 1 comment 1 reply

Comment options

You must be logged in to vote
1 reply
@Ghjk94522
Comment options

Answer selected by Ghjk94522
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants