No hardware acceleration could be configured, running in CPU mode

[quanpy]

why i get the newest docker image: beveradb/audio-separator:gpu-0.12.3, when i run in the docker，GPU can not be used。

2024-01-23 07:37:54.481 - INFO - separator - Separator version 0.13.0 instantiating with output_dir: /workdir/separated, output_format: WAV
2024-01-23 07:37:54.481 - INFO - separator - Checking hardware specifics to configure acceleration
2024-01-23 07:37:54.481 - INFO - separator - Operating System: Linux #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023
2024-01-23 07:37:54.481 - INFO - separator - System: Linux Node: 2268b32cf6d1 Release: 5.15.0-91-generic Machine: x86_64 Proc: x86_64
2024-01-23 07:37:54.481 - INFO - separator - Python Version: 3.10.12
2024-01-23 07:37:54.481 - INFO - separator - ONNX Runtime GPU package installed with version: 1.16.3
2024-01-23 07:37:54.490 - INFO - separator - Torch package installed with version: 2.1.2
2024-01-23 07:37:54.490 - INFO - separator - Torchvision package installed with version: 0.16.2
2024-01-23 07:37:54.490 - INFO - separator - Torchaudio package installed with version: 2.1.2
2024-01-23 07:37:54.513 - INFO - separator - No hardware acceleration could be configured, running in CPU mode

[quanpy]

2024-01-23 08:00:34.497 - INFO - separator - Torchaudio package installed with version: 2.1.2
/opt/conda/lib/python3.10/site-packages/torch/cuda/init.py:138: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at /opt/conda/conda-bld/pytorch_1702400410390/work/c10/cuda/CUDAFunctions.cpp:108.)
return torch._C._cuda_getDeviceCount() > 0
2024-01-23 08:00:34.499 - DEBUG - separator - CUDA not available in Torch installation. If you expect GPU/CUDA support to work, please see README

[beveradb]

Hmm, that's strange, I've tested that specific docker image on runpod (I don't have a GPU of my own unfortunately)

I wonder if you need to pass in the GPU device to the docker container when running it, perhaps?

Does e.g. docker run --gpus all nvidia/cuda:11.0-base nvidia-smi work?

[quanpy]

it worked, but I find that the cuda version is 12.2.2 on my host machine, which in container is 12.2.3. This may cause this problem, thanks.

[beveradb]

Huh, I'd actually be surprised if that was an issue 🤔

Have you tried the CUDA 11 docker image? docker pull beveradb/audio-separator:gpu
Dockerfile: https://github.com/karaokenerds/python-audio-separator/blob/main/Dockerfile.gpu

The reason I ask is just that CUDA 12 is still not officially supported by the ONNX Runtime (though that should change in the next few days, v 1.17 finally has release candidates: microsoft/onnxruntime#19236 (comment) )

If you figure out how to get it working, or if you have any suggestions for how to improve the Dockerfiles or README instructions in this repo, please share! 🙇

[beveradb]

FYI @quanpy , this should now be resolved with the latest versions of audio-separator (version 0.14.4 or newer), as it's now using the newly-released ONNX runtime version 1.17 which supports CUDA 12!

Please give it a try and if you're still having issues with the latest version, share your full log output from audio-separator in a new issue 🙇