Skip to content

Commit d8f785a

Browse files
committed
[V1] Enhance check when clicing encoder output
Prepare for vllm-project#11409 For pixtral model, we need to insert placeholders in the middle of encoder output, to fit into whole soft embedding. This case makes slicing operation tricky. This PR raises assertion if something's off. Signed-off-by: Linkun Chen <[email protected]>
1 parent 6077919 commit d8f785a

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

vllm/v1/worker/gpu_model_runner.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -734,6 +734,7 @@ def _gather_encoder_outputs(
734734
assert req_id in self.encoder_cache
735735
assert i in self.encoder_cache[req_id]
736736
encoder_output = self.encoder_cache[req_id][i]
737+
assert end_idx <= encoder_output.shape[0], f"{end_idx=} {encoder_output.shape=}"
737738
encoder_outputs.append(encoder_output[start_idx:end_idx])
738739
return encoder_outputs
739740

0 commit comments

Comments
 (0)