Does xformers work on AMD GPU?

[Chryseus]

I've had no luck getting it working on Arch Linux, I dunno if it's because of a problem with Arch or that it just doesn't work on AMD hardware.

[ClashSAN]

no, you will not be able to install from pre-compiled xformers wheels.
--xformers flag will install for Pascal, Turing, Ampere, Lovelace or Hopper NVIDIA cards.

[Chryseus]

That isn't an issue for me, I'm perfectly happy to compile it, I just dunno if my current problems are due to Arch or using AMD.

[ClashSAN]

I haven't seen this, so idk
maybe there is an amd rentry guide that has some info.
so if you do get it working, share stuff

[Boom-Hacker]

no it not work on my rx 5500xt,output:NotImplementedError: No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 6144, 1, 512) (torch.float32)
key : shape=(1, 6144, 1, 512) (torch.float32)
value : shape=(1, 6144, 1, 512) (torch.float32)
attn_bias : <class 'NoneType'>
p : 0.0
cutlassF is not supported because:
xFormers wasn't build with CUDA support
flshattF is not supported because:
xFormers wasn't build with CUDA support
dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})
max(query.shape[-1] != value.shape[-1]) > 128
tritonflashattF is not supported because:
xFormers wasn't build with CUDA support
dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})
max(query.shape[-1] != value.shape[-1]) > 128
triton is not available
requires A100 GPU
smallkF is not supported because:
xFormers wasn't build with CUDA support
max(query.shape[-1] != value.shape[-1]) > 32
unsupported embed per head: 512

[rakataprime]

I think you would have to hipify xformers to make it work on rocm

[BahzBeih]

so did anyone did this already? or it's so difficult.

[GoZippy]

any ideas where to look?

[InCrIpTiOn]

You will need to wait for torch to be updated with HIP support, which requires the propeitery software like MIOpen. You can see progress here.

[AR2000AR]

xformers is availables for rocm : https://rocm.docs.amd.com/en/latest/how-to/rocm-for-ai/inference-optimization/model-acceleration-libraries.html#xformers

[Soulreaver90]

Did you get it to work? There was another discussion thread and I was unable to get it working when I had tried it several weeks ago. I am aware they recently pushed out an update with better support but haven't tested it out yet.

[AR2000AR]

Unfortunately no. It requires torch2.6.0 but to update it something break in torchvision and the app won't start