Skip to content

Commit 5a7038d

Browse files
committed
Add TORCH_CHECKs
Signed-off-by: Vadim Gimpelson <[email protected]>
1 parent 7032f7c commit 5a7038d

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/_custom_ops.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,7 @@ def fused_add_rms_norm(input: torch.Tensor, residual: torch.Tensor,
280280
weight: torch.Tensor, epsilon: float) -> None:
281281
torch.ops._C.fused_add_rms_norm(input, residual, weight, epsilon)
282282

283-
283+
@torch.compile
284284
def apply_repetition_penalties_torch(
285285
logits: torch.Tensor, prompt_mask: torch.Tensor,
286286
output_mask: torch.Tensor, repetition_penalties: torch.Tensor) -> None:

0 commit comments

Comments
 (0)