Skip to content

Commit ddb9d85

Browse files
yiyixuxusayakpaulstevhliu
authored
[doc] add a tip about using SDXL refiner with hunyuan-dit and pixart (#8735)
* up * Apply suggestions from code review Co-authored-by: Steven Liu <[email protected]> --------- Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: Steven Liu <[email protected]>
1 parent 4997975 commit ddb9d85

File tree

4 files changed

+19
-1
lines changed

4 files changed

+19
-1
lines changed

docs/source/en/api/pipelines/hunyuandit.md

+6
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,12 @@ Make sure to check out the Schedulers [guide](../../using-diffusers/schedulers.m
3434

3535
</Tip>
3636

37+
<Tip>
38+
39+
You can further improve generation quality by passing the generated image from [`HungyuanDiTPipeline`] to the [SDXL refiner](../../using-diffusers/sdxl#base-to-refiner-model) model.
40+
41+
</Tip>
42+
3743
## Optimization
3844

3945
You can optimize the pipeline's runtime and memory consumption with torch.compile and feed-forward chunking. To learn about other optimization methods, check out the [Speed up inference](../../optimization/fp16) and [Reduce memory usage](../../optimization/memory) guides.

docs/source/en/api/pipelines/pixart_sigma.md

+6
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,12 @@ Make sure to check out the Schedulers [guide](../../using-diffusers/schedulers)
3737

3838
</Tip>
3939

40+
<Tip>
41+
42+
You can further improve generation quality by passing the generated image from [`PixArtSigmaPipeline`] to the [SDXL refiner](../../using-diffusers/sdxl#base-to-refiner-model) model.
43+
44+
</Tip>
45+
4046
## Inference with under 8GB GPU VRAM
4147

4248
Run the [`PixArtSigmaPipeline`] with under 8GB GPU VRAM by loading the text encoder in 8-bit precision. Let's walk through a full-fledged example.

docs/source/en/using-diffusers/sdxl.md

+6
Original file line numberDiff line numberDiff line change
@@ -285,6 +285,12 @@ refiner = DiffusionPipeline.from_pretrained(
285285
).to("cuda")
286286
```
287287

288+
<Tip warning={true}>
289+
290+
You can use SDXL refiner with a different base model. For example, you can use the [Hunyuan-DiT](../../api/pipelines/hunyuandit) or [PixArt-Sigma](../../api/pipelines/pixart_sigma) pipelines to generate images with better prompt adherence. Once you have generated an image, you can pass it to the SDXL refiner model to enhance final generation quality.
291+
292+
</Tip>
293+
288294
Generate an image from the base model, and set the model output to **latent** space:
289295

290296
```py

src/diffusers/pipelines/stable_diffusion_3/pipeline_stable_diffusion_3_img2img.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@
6262
>>> pipe = pipe.to(device)
6363
6464
>>> url = "https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg"
65-
>>> init_image = load_image(url).resize((512, 512))
65+
>>> init_image = load_image(url).resize((1024, 1024))
6666
6767
>>> prompt = "cat wizard, gandalf, lord of the rings, detailed, fantasy, cute, adorable, Pixar, Disney, 8k"
6868

0 commit comments

Comments
 (0)