Skip to content

Commit 01bd796

Browse files
hlkysayakpaul
andauthored
Fix HunyuanVideo produces NaN on PyTorch<2.5 (#10482)
Co-authored-by: Sayak Paul <[email protected]>
1 parent 03bcf5a commit 01bd796

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

Diff for: src/diffusers/models/transformers/transformer_hunyuan_video.py

+4-4
Original file line numberDiff line numberDiff line change
@@ -713,15 +713,15 @@ def forward(
713713
condition_sequence_length = encoder_hidden_states.shape[1]
714714
sequence_length = latent_sequence_length + condition_sequence_length
715715
attention_mask = torch.zeros(
716-
batch_size, sequence_length, sequence_length, device=hidden_states.device, dtype=torch.bool
717-
) # [B, N, N]
716+
batch_size, sequence_length, device=hidden_states.device, dtype=torch.bool
717+
) # [B, N]
718718

719719
effective_condition_sequence_length = encoder_attention_mask.sum(dim=1, dtype=torch.int) # [B,]
720720
effective_sequence_length = latent_sequence_length + effective_condition_sequence_length
721721

722722
for i in range(batch_size):
723-
attention_mask[i, : effective_sequence_length[i], : effective_sequence_length[i]] = True
724-
attention_mask = attention_mask.unsqueeze(1) # [B, 1, N, N], for broadcasting across attention heads
723+
attention_mask[i, : effective_sequence_length[i]] = True
724+
attention_mask = attention_mask.unsqueeze(1) # [B, 1, N], for broadcasting across attention heads
725725

726726
# 4. Transformer blocks
727727
if torch.is_grad_enabled() and self.gradient_checkpointing:

0 commit comments

Comments
 (0)