Attend and Excite

[patrickvonplaten]

Model/Pipeline/Scheduler description

Based on Prompt-2-Prompt, Attend and Excite allows to generate better composite images by tweaking the text embeddings.

Code available: https://github.com/AttendAndExcite/Attend-and-Excite

Open source status

• The model implementation is available
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

@AttendAndExcite

[patrickvonplaten]

Related to: #2121

[evinpinar]

I've created a pipeline for this method, following the author's implementation closely, and also using the attention processor, in half precision. See here: yuval-alaluf/Attend-and-Excite#11

Please let me know how to contribute to diffusers repo with this pipeline, also if any modifications are needed.

[patrickvonplaten]

Super cool @evinpinar :-)

Could you maybe open a PR on diffusers? @yiyixuxu would you be interested in reviewing it maybe? :-)

[adhikjoshi]

This is awesome, looking forward to quick integration.

Will this support current schedulers and "custom_pipeline" ?

[evinpinar]

Super cool @evinpinar :-)

Could you maybe open a PR on diffusers? @yiyixuxu would you be interested in reviewing it maybe? :-)

Now I started a PR here, but as I'm quite busy nowadays, open to collaborating with anyone!

[yiyixuxu]

Super cool @evinpinar :-)
Could you maybe open a PR on diffusers? @yiyixuxu would you be interested in reviewing it maybe? :-)

Now I started a PR here, but as I'm quite busy nowadays, open to collaborating with anyone!

Thank you! will take a look soon:)

[hamzafar]

I was wondering, Is it possible to integrate the attend and excite code with Hugging Face diffusers? I trained a stable diffusion model with the Hugging Face diffusers GitHub code and used the saved model with attend and excite, which worked flawlessly. Thought of, sharing the Colab notebook for the community showing the read model trained with Hugging Face diffusers with attend and excite: https://colab.research.google.com/drive/1xpgXZf0SxCxVw77GcgLxBr6XC8gItFMh?usp=sharing

[patrickvonplaten]

That's great @hamzafar ! Thanks for sharing :-)

[adhikjoshi]

As i can see i can use pipeline for text2img,

Any example of how can i use with img2img as well? If it's possible at all?

[a-r-r-o-w]

As SD attend-and-excite exists as a pipeline now, but also very outdated, marking this as closed. Thanks for looking into it when you did Patrick! It was super nice to learn and understand