use attention mask parameter in flux attention

[christopher5106]

Unused params attention_mask in transformer flux attention processors is useful to reweight prompt or image prompt.

Fixes #10025

@yiyixuxu @sayakpaul

[bghira]

this is incomplete. more changes are required to make use of the text encoder attn masking since it's being combined with the image tokens.

[bonlime]

Thumbs up to this request. Diffusers is not only a set of pipelines, but also a library of use-full components, even if currently there is no diffusers pipeline that uses attention masking for flux, it doesn't mean it shouldn't be passed. I personally have a flux pipeline that requires passing attention mask and currently had to copy-paste existing attn processor only to start using this argument

[bghira]

i have code to do attn masking in flux and would like it to have feature parity with what Diffusers will do. hence this is incomplete.there is more than just passing args in using generic parameter names. you can use image_attention_mask and encoder_attention_mask and fill with ones if encoder mask is supplied but an image mask isnt.

however i think more importantly there is a fundamental misapplication of attention masking happening here, because it is not for re-weighting inputs.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[bonlime]

This has been closed by #10122

[bghira]

i'll just keep mentioning that it's an incomplete fix that has been merged and that this is not yet resolved

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.