[fbsync] fix Swin Transformer inplace mutation (#6266)

Summary:
* fix inplace mutation

* Different attn shouldn't share the same attribute

* a simpler solution

Reviewed By: jdsgomes

Differential Revision: D37993419

fbshipit-source-id: 2a08a62168c4e6ee6c5a2ca934de88aa04361016

Co-authored-by: YosuaMichael <[email protected]>

Author: 2 people

torchvision/models/swin_transformer.py

@@ -106,6 +106,7 @@ def shifted_window_attention(
     x = F.pad(input, (0, 0, 0, pad_r, 0, pad_b))
     _, pad_H, pad_W, _ = x.shape
 
+    shift_size = shift_size.copy()
     # If window size is larger than feature size, there is no need to shift window
     if window_size[0] >= pad_H:
         shift_size[0] = 0