RuntimeWarning: invalid value encountered in cast images = (images * 255).round().astype("uint8") output black image #6815
Labels
bug
Something isn't working
Comments
|
Hi @sayakpaul @yiyixuxu, I would like to work on this issue. Please let me know how can I start. Thanks |
|
Same on my GTX 1660 Ti with CUDA 12.3. Replace Same for webui using:
Before:
After: Source: # https://huggingface.co/nerijs/pixel-art-xl
import time
from diffusers import DiffusionPipeline, LCMScheduler
import torch
model_id = "stabilityai/stable-diffusion-xl-base-1.0"
lcm_lora_id = "latent-consistency/lcm-lora-sdxl"
pipe = DiffusionPipeline.from_pretrained(model_id, variant="fp16")
pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)
pipe.load_lora_weights(lcm_lora_id, adapter_name="lora")
# Path not found? huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-65bf9aae-1bf2b36602e3f1251ea11e7d;4af0f325-1b83-40e4-9cc1-262f5eb8f401)
pipe.load_lora_weights("pixel-art-xl.safetensors", adapter_name="pixel")
pipe.set_adapters(["lora", "pixel"], adapter_weights=[1.0, 1.2])
pipe.to(device="cuda", dtype=torch.float32) # float16 results in black images after
# d:\code\AI\Python>python du_lora.py
# C:\Users\C\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
# torch.utils._pytree._register_pytree_node(
# Loading pipeline components...: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:11<00:00, 1.68s/it]
# The config attributes {'skip_prk_steps': True} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file.
# 0%| | 0/8 [00:00<?, ?it/s]C:\Users\C\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\models\attention_processor.py:1244: UserWarning: 1Torch was not compiled with flash attention. (Triggered internally at ..\aten\src\ATen\native\transformers\cuda\sdp_utils.cpp:263.)
# hidden_states = F.scaled_dot_product_attention(
# 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [08:23<00:00, 62.96s/it]
# C:\Users\C\AppData\Local\Programs\Python\Python310\lib\site-packages\diffusers\image_processor.py:90: RuntimeWarning: invalid value encountered in cast
# images = (images * 255).round().astype("uint8")
prompt = "pixel, a cute corgi"
negative_prompt = "3d render, realistic"
num_images = 9
for i in range(num_images):
img = pipe(
prompt=prompt,
negative_prompt=negative_prompt,
num_inference_steps=8,
guidance_scale=1.5,
).images[0]
img.save(f"{time.strftime('%Y-%m-%d %H%M%S')}_lcm_lora_{i}.png") |
|
Hi @CTimmerman, does this problem exists for torch.float16 on a windows system? Should the solution be crafted? Thanks |
|
@Bhavay-2001 Yes: |
|
I think we need to discuss this with @sayakpaul @yiyixuxu and then I can contribute in this. |
|
Enabling autocast is better than raising the error to the user and outputting black images, but i think that suddenly stopped working in my webgui until i started using |
|
Soo how can we make this change? Any helps will be appreciated. |
|
We have made a number of fixes to the DPM schedulers. Could you try installing the latest version of Cc: @yiyixuxu |
Main doesn't look any different, and there are too many other branches to read. If this is not a problem limited to 16xx cards, then you can test it using my code by changing I'm already using the main branch: That code is not used in this issue. |
|
how can I fix the problem though this has also been happening with the upscaler models too. When I run it in google colab it works. |
|
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
|
Gently pinging @yiyixuxu. |
|
I'm not sure what to do here other than use full precision |
|
I think I'm seeing this same issue with on a 4090 on ubuntu, using the train_text_to_image_lora_sdxl script dropping lr to 1e-6 doesn't help. The images passed to tensorboard are black. Could be other default parameters are causing the issue. The same dataset works fine the the sd1.5 lora trainer. |
|
You might want to use a more numerically stable VAE: "madebyollin/sdxl-vae-fp16-fix" and pass it to |
|
Thanks @sayakpaul that solves the problem for me |
|
扩散模型的预训练权重在推理时会发生变化，重新下载预训练权重即可解决问题 |
|
Just in case someone else chances upon this thread... |
|
It worked after using fp32, but we need a hero explain why it failed and how to make it work with fp16？ thx |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
When running the stable-diffusion-2-1 I get a runtime warning "RuntimeWarning: invalid value encountered in cast
images = (images * 255).round().astype("uint8")" and the image output is black.
Reproduction
My code:
import torch
from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler
model_id = "stabilityai/stable-diffusion-2-1"
comment#Use the DPMSolverMultistepScheduler (DPM-Solver++) scheduler here instead
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
pipe = pipe.to("cuda")
prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt).images[0]
image.save("astronaut_rides_horse.png")
I got it from here: https://huggingface.co/stabilityai/stable-diffusion-2-1
Logs
System Info
diffusersversion: 0.26.0Who can help?
No response
The text was updated successfully, but these errors were encountered: