Skip to content

Commit cd341cf

Browse files
gau-nernstshreyankg
authored andcommitted
[Bugfix] Fix cache block size calculation for CPU MLA (vllm-project#15848)
Signed-off-by: Thien Tran <[email protected]>
1 parent a914c04 commit cd341cf

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/worker/cpu_worker.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ def get_cache_block_size(
111111
num_layers = model_config.get_num_layers(parallel_config)
112112

113113
key_cache_block = block_size * num_heads * head_size
114-
value_cache_block = key_cache_block
114+
value_cache_block = key_cache_block if not model_config.use_mla else 0
115115
total = num_layers * (key_cache_block + value_cache_block)
116116
if cache_dtype == "auto":
117117
dtype = model_config.dtype

0 commit comments

Comments
 (0)