Skip to content

Latest commit

 

History

History
48 lines (30 loc) · 3.45 KB

README.md

File metadata and controls

48 lines (30 loc) · 3.45 KB

ComfyUI Backend Extension For SwarmUI

This extension enables the use of ComfyUI as a backend provider for SwarmUI.

Among other benefits, this enables you to use custom ComfyUI-API workflow files within SwarmUI.

You can also view the ComfyUI node graph and work with custom workflows directly in the UI when any comfy backend is enabled.

API vs Self-Start

  • Self-Start lets swarm configure, launch, and manage the ComfyUI backend. This is highly recommended.
  • API-By-URL is for if you want to launch and manage the ComfyUI instance entirely yourself, but still connect it from Swarm.
    • Configuration is significantly more complex, and misbehavior may occur. This is not recommended.
    • In other words: Unless you're a professional getting paid by the hour to build a complex AI network, you almost certainly do not need or want the API-By-URL option.

Installation (Self-Start)

  • First: Have a valid ComfyUI install. The SwarmUI installer automatically provides you one (if not disabled) as dlbackend/comfy/ComfyUI/main.py.
  • Go to Server -> Backends, and click ComfyUI Self-Starting, and fill in the StartScript path as above. Other values can be left default or configured to your preference.
  • Save the backend, and it should just work.

Installation (API)

  • First: have a valid and working ComfyUI installation.
  • Make sure it uses the exact same model paths as your SwarmUI instance does. This means that if you have eg OfficialStableDiffusion/sd_xl_base_1.0.safetensors in Swarm, you need have EXACTLY that in ComfyUI. The only exception is Windows paths that use \ instead of / are fine, Swarm will automatically correct for that (If you use Self-Start, this is automatically managed from your Swarm settings).
  • Note that swarm may leave stray Input or Output images in the ComfyUI folder that you may wish to clean up (if you use Self-Start, this will be prevented automatically).
  • Swarm provides extra Comfy nodes automatically to Self-Start ComfyUI instances from folders within the ComfyUI extension folder, including DLNodes and ExtraNodes - it is highly recommended you copy these to your remote Comfy's custom_nodes path, or point to them with your extra_model_paths file.
  • If you use a Self-Start backend, it will autogenerate a valid extra model paths file into the Data folder, you may wish to do that to copy for your comfy API instance.

Basic Usage Within SwarmUI

(TODO): tldr don't worry about it, it just works, follow general Swarm usage docs

Using Workflows In The UI

(TODO): explain the Node tab and how to use it within SwarmUI, link out to Comfy docs for usage of the node editor itself.

  • When using a custom workflow in the main Generate tab:
    • Default nodes (KSampler, LoadCheckpoint, etc) will automatically detect and link to standard Swarm workflows.
    • You can use the SwarmLoraLoader node to allow loading loras in your workflow, see here

Making and Using Your Own Custom Workflow Files

(TODO): explain the API-specific workflow file format, how it differs from workflows in the UI, and how to use it.

(TODO): Are API-format custom workflows even relevant anymore? UI-workflows are easier and nicer.

(Note: this readme section should mention that the main checkpoint loader should be ID 4 for best compatibility, due to how ComfyUI loads models - see just_load_model.json)