Installation on Intel Silicon

Stable Diffusion WebUI Support (Preview)

Stable Diffusion WebUI can now be run on hardware like Intel CPUs and GPUs (both integrated and discrete Graphics) with Intel® Distribution of OpenVINO™ toolkit. This is a preview support in active development and we actively welcome feedback and contributions from the community.

Instructions to run WebUI with OpenVINO:

OpenVINO support is provided through a custom script. The custom script uses PyTorch's torch.compile feature and HuggingFace Diffusers library for improved performance. Below are the instructions to get started:

• If you are familiar with the Automatic1111 workflow, use this fork from OpenVINOToolKit instead of Automatic1111 and follow the instructions. Alternatively, follow the below instructions:

Linux

# Make sure Python version is 3.10+
python -m venv sd_env
source sd_env/bin/activate
git clone -b openvino_custom_scripts https://github.com/openvinotoolkit/stable-diffusion-webui.git
cd stable-diffusion-webui

export PYTORCH_TRACING_MODE=TORCHFX
export COMMANDLINE_ARGS="--skip-torch-cuda-test --precision full --no-half --listen --share" 

• Install the prerequisites by following the instructions here
• Install OpenVINO in the same python environment by following the instructions here
• Set the environment variable COMMANDLINE_ARGS to "--skip-torch-cuda-test --precision full --no-half" .
• To access WebUI through a public URL from a remote machine, add additional arguments to COMMANDLINE_ARGS "--listen --share"
• Launch the WebUI by running webui.sh
• Use the OpenVINO custom script by following the instructions here

Windows

python -m venv sd_env
sd_env\Scripts\activate
git clone -b openvino_custom_scripts https://github.com/openvinotoolkit/stable-diffusion-webui.git
cd stable-diffusion-webui

• Install OpenVINO in the same python environment by following the instructions here
• In stable-diffusion-webui directory, update webui-user.bat file so it should look like below

@echo off

set PYTHON=
set GIT=
set VENV_DIR=-
set COMMANDLINE_ARGS=--skip-torch-cuda-test --precision full --no-half
set PYTORCH_TRACING_MODE=TORCHFX

call webui.bat

• Launch the WebUI for the first time by running webui-user.bat. It should install the prerequisites.
• Once it is done installing all prerequisites, terminate WebUI using CTRL+C.
• Install the additional prerequisites by following the instructions here
• Add --skip-prepare-environment argument to the COMMANDLINE_ARGS variable in webui-user.bat file. To access WebUI through a public URL from a remote machine, add additional arguments to COMMANDLINE_ARGS "--listen --share"
• Launch the WebUI again by running webui-user.bat
• Use the OpenVINO custom script by following the instructions here

Installation Instructions

Prerequisites

Diffusers

We use the diffusers library for improved performance with torch.compile. The recommended version is >= v0.17.0. Diffusers can be installed from PyPi: pip install diffusers

PyTorch

OpenVINO is integrated as a TorchDynamo backend in PyTorch and can be leveraged using "torch.compile". TorchDynamo is available from PyTorch 2.0 onwards.

Linux:

Install PyTorch from PyPi using pip install torch

Windows:

PyTorch doesn't support torch.compile officially on windows yet. However, the below workaround can be used to leverage torch.compile feature on windows:

• Install the validated PyTorch nightly wheel file (2.1.0.dev20230713) from here
• Update the file at <virtual_env_root>/Lib/site-packages/torch/_dynamo/eval_frames.py
• Find the function called "check_if_dynamo_supported()" which looks like this (line 484):

def check_if_dynamo_supported():
    if sys.platform == "win32":
        raise RuntimeError("Windows not yet supported for torch.compile")
    if sys.version_info >= (3, 11):
        raise RuntimeError("Python 3.11+ not yet supported for torch.compile")

• Comment out the first two lines of this function. It should look like this:

def check_if_dynamo_supported():
    #if sys.platform == "win32":
    #    raise RuntimeError("Windows not yet supported for torch.compile")
    if sys.version_info >= (3, 11):
        `raise RuntimeError("Python 3.11+ not yet supported for torch.compile")

Install OpenVINO:

Install from Pypi:

OpenVINO with torch.compile support is now available in preview in the OpenVINO prerelease package (version: Add Version). Download the prerelease package from here. Install from Pypi using the command pip install --pre openvino

Build and install from source:

OpenVINO can also be built from source using the instructions provided here

Known Issues

• Changing the sampling method to DPM++ or Karras methods recompiles the model due to some modifications it makes to the graph. It is recommended to exclude the time for first image generation for any performance measurements