Skip to content

Commit c1f8148

Browse files
varun-sundar-rabindranathVarun Sundar Rabindranath
authored andcommitted
[Bugfix] Fix _get_lora_device for HQQ marlin (vllm-project#12090)
Signed-off-by: Varun Sundar Rabindranath <[email protected]> Co-authored-by: Varun Sundar Rabindranath <[email protected]>
1 parent 85af62f commit c1f8148

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

vllm/lora/layers.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,9 @@ def _get_lora_device(base_layer: nn.Module) -> torch.device:
5151
# marlin
5252
elif hasattr(base_layer, "B"):
5353
return base_layer.B.device
54+
# HQQ marlin
55+
elif hasattr(base_layer, "W_q"):
56+
return base_layer.W_q.device
5457
else:
5558
raise ValueError(f"Unsupported base layer: {base_layer}")
5659

0 commit comments

Comments
 (0)