Generated images contain a green background with foreground objects

[baarongit]

Describe the bug

Hi,

For any input image and any prompt, I am continually getting green backgrounds.

I am using the CompVis/stable-diffusion-v1-4 model to generate an image to image with a prompt similar to:

Abstract pixel art of gramophone with blue background

With many different variations of strength and image sizes, the result is the same, green backgrounds.

The code is as follows:

if torch.cuda.is_available():
                print("Using CUDA")
                self._pipe_line = StableDiffusionImg2ImgPipeline.from_pretrained(self._model_id,cache_dir=self._cache_dir,revision="fp16", torch_dtype=torch.float16, use_auth_token=True).to(ImageSystem.DEVICE)
                self._pipe_line.safety_checker = None
                #self._pipe_line.enable_attention_slicing() 
            else:
                print("Using CPU")
                self._pipe_line  = StableDiffusionImg2ImgPipeline.from_pretrained(self._model_id,cache_dir=self._cache_dir).to(ImageSystem.DEVICE)
            
generator = torch.Generator(device=ImageSystem.DEVICE).manual_seed(self._seed)
            
with autocast("cuda"):
                    result_image = self._pipe_line(
                        instruction,
                        image = image, # The starting image
                        strength = self._strength, # 0 for no change, 1.0 for max strength
                        #num_inference_steps=30
                        generator=generator
                    ).images[0]

The machine has a V100 GPU with 32GB of memory. The CUDA version is:

NVIDIA-SMI 525.116.03 Driver Version: 525.116.03 CUDA Version: 12.0

Reproduction

if torch.cuda.is_available():
                print("Using CUDA")
                self._pipe_line = StableDiffusionImg2ImgPipeline.from_pretrained(self._model_id,cache_dir=self._cache_dir,revision="fp16", torch_dtype=torch.float16, use_auth_token=True).to(ImageSystem.DEVICE)
                self._pipe_line.safety_checker = None
                #self._pipe_line.enable_attention_slicing() 
            else:
                print("Using CPU")
                self._pipe_line  = StableDiffusionImg2ImgPipeline.from_pretrained(self._model_id,cache_dir=self._cache_dir).to(ImageSystem.DEVICE)
            
generator = torch.Generator(device=ImageSystem.DEVICE).manual_seed(self._seed)
            
with autocast("cuda"):
                    result_image = self._pipe_line(
                        instruction,
                        image = image, # The starting image
                        strength = self._strength, # 0 for no change, 1.0 for max strength
                        #num_inference_steps=30
                        generator=generator
                    ).images[0]

Logs

No response

System Info

The machine has a V100 GPU with 32GB of memory. The CUDA version is:

NVIDIA-SMI 525.116.03 Driver Version: 525.116.03 CUDA Version: 12.0

The OS is:

Linux version 4.18.0-425.19.2.el8_7.x86_64 ([email protected]) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-16) (GCC)) #1 SMP Fri Mar 17 01:52:38 EDT 2023

Who can help?

No response

[ghost]

I am having the same issues as above and need this worked out for urgent client delivery.

[gibm-a]

Hello folks, I'm having the same issue as the above commenter, it would be great if we can get some help.

[yiyixuxu]

Hi:

can you provide inputs along with the script so that I can reproduce on my end?

Thanks

YiYi

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[tolgacangoz]

Hi @baarongit, @gibm-a. Could you provide us with a reproducible notebook?
@baarongit, I tried your code but there was not a green background issue.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.