Skip to content

Commit 88c07ee

Browse files
jeejeeleeIsotr0py
authored andcommitted
[Misc] Improve the readability of BNB error messages (vllm-project#12320)
Signed-off-by: Jee Jee Li <[email protected]> Signed-off-by: Isotr0py <[email protected]>
1 parent eb0f4a5 commit 88c07ee

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

vllm/model_executor/model_loader/loader.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1076,8 +1076,8 @@ def _load_weights(self, model_config: ModelConfig,
10761076
# weight tensor. So TP does not work with pre_quantized bnb models.
10771077
if pre_quant and get_tensor_model_parallel_world_size() > 1:
10781078
raise ValueError(
1079-
"Prequant BitsAndBytes models with TP is not supported."
1080-
"Please try with PP.")
1079+
"Prequant BitsAndBytes models with tensor parallelism is not "
1080+
"supported. Please try with pipeline parallelism.")
10811081

10821082
load_8bit = False
10831083
if pre_quant:

0 commit comments

Comments
 (0)