Multi GPU Systemd w Templates

The problem

Do you have multiple GPUs within a single server and wanted to start and stop them easily with systemd? With systemd templates this can be easily accomplished. There are a few steps to get this running.

The solution

This solution presumes you are using 2 or more Nvidia GPUs within a single server. This may work with Intel or AMD GPUs but may require tweaking for the Device IDs. Additionally each worker instance port number will be increased by 1. All worker instances will have the same models and extensions so be sure to check if this will cause issues running multiple copies from the same folder.

Copy and edit webui startup files

First create a copy of the webui.sh and webui-user.sh in the root of your webui directory. In this instances we will call them webui-worker.sh and webui-user-worker.sh. You will need to remove any of the --distributed-* arguments from the webui-user-worker.sh

webui-worker.sh

Replace the second block with the following code

# Read variables from webui-user-worker.sh
# shellcheck source=/dev/null
if [[ -f webui-user-worker.sh ]]
then
    source ./webui-user-worker.sh
fi

webui-user-worker.sh

Add the following lines after clone_dir

offset=$CUDA_VISIBILE_DEVICES

port=$((offset + 7860))

and add these lines to the end of the COMMANDLINE_ARGS --port $port

Create systemd template service

Now we can create a systemd template service that will form the basis of our worker instances. By including the @ in the service name denotes to systemd that this is a template service that will take information after the @ and replace variables in the service instance Create a new systemd service template with the following command.

sudo systemctl edit --full --force [email protected]

Place the following code inside and customize the username and path as needed to your webui folder.

[Unit]
Description=WebUI Service GPU %i
After=network.target

[Service]
Type=simple
User=<your username>
Restart=always
RestartSec=1
Environment="CUDA_VISIBILE_DEVICES=%i"
ExecStart=/usr/bin/env bash /opt/stabled/webui-worker.sh 
StandardOutput=append:/var/log/sdwebui-%i.log
StandardError=append:/var/log/sdwebui-%i.log
WorkingDirectory=/opt/stabled

[Install]
WantedBy=multi-user.target

Start worker instances

Starting a worker instance is as easy as the following command. Change the 1 any number of GPUs that you have within the system.

sudo systemctl start [email protected]

And for those of you with copious amounts of GPUs here is a slick bash one liner to start all of them. Replace 1..3 with the range of secondary GPUs that you have in the service.

sudo bash -c 'for i in {1..3}; do systemctl start stabled-worker@"$i".service; done'

If you want these to persist on reboot you can enable them as needed with

sudo systemctl enable [email protected]