v0.8.2 "No available memory for the cache blocks." when launching models on the same GPU

[CaoChensy]

Description:
I'm encountering an issue when trying to serve two different models using vLLM. The behavior differs between the v1 engine and when forcing v0 engine via VLLM_USE_V1=0.

1. Both models work fine when using VLLM_USE_V1=0:

VLLM_USE_V1=0 vllm serve /home/models/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B/ --port 8101 --tensor-parallel-size 1 --max-model-len 8192 --enforce-eager --gpu-memory-utilization 0.5

VLLM_USE_V1=0 vllm serve /home/models/Qwen/Qwen2.5-14B-Instruct --port 8102 --tensor-parallel-size 1 --max-model-len 8192 --enforce-eager --gpu-memory-utilization 0.45

2. However, when running without VLLM_USE_V1=0:

   • DeepSeek-R1-Distill-Qwen-14B loads normally
   • Qwen2.5-14B-Instruct fails with a memory error:

ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

Environment:

• vLLM version: 0.8.2
• Python version: 3.12

Additional Context:

• The Qwen2.5 model fails to initialize KV cache in v1 engine despite working with v0 engine at lower memory utilization (0.45 vs 0.5)
• Both models are 14B parameter size, and GPU memory 80GB
• Using --enforce-eager mode
• Tensor parallelism size is 1 for both
• Max model length set to 8192 for both

Questions:

1. Why does the v1 engine require more memory for the same model compared to v0?
2. Is there a known issue with Qwen2.5 models in the v1 engine?
3. Are there additional parameters that could help the v1 engine allocate memory more efficiently?

[reallm]

same issue !!!

[DarkLight1337]

The default value of --max-num-seqs has been increased from 256 in V0 to 1024 in V1 which results in higher memory usage by default. You can try setting the --max-num-seqs to a lower value.