Build llama.cpp for consoles

[tejasAMGI]

Hi, is there a way to generate .libs using cmake for consoles like XBOX and PS? Since consoles have unified memory and different build toolchain, the CPU build may not work, right?

[rgerganov]

I have successfully built and run llama.cpp on Steam Deck with Vulkan backend. Will share the details when I have some time. Steam Deck is running Linux, so it's a kind of straightforward there.

[tejasAMGI]

Thats great! I am assuming you built shared objects (.so) files and used them, right?

[rgerganov]

Yes, I compiled everything on my Ubuntu 24.04 workstation and then simply copied all binaries and libs to the console. libvulkan.so is already in /usr/lib

(deck@steamdeck vulkan)$ ldd ./llama-bench 
	linux-vdso.so.1 (0x00007ffd90ddd000)
	libllama.so => /home/deck/llamacpp/vulkan/libllama.so (0x00007f5278d3b000)
	libggml.so => /home/deck/llamacpp/vulkan/libggml.so (0x00007f5278d2f000)
	libggml-base.so => /home/deck/llamacpp/vulkan/libggml-base.so (0x00007f5278c59000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f5278800000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007f5278b4a000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f5278b23000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f527861e000)
	/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f5278f38000)
	libggml-cpu.so => /home/deck/llamacpp/vulkan/libggml-cpu.so (0x00007f5278545000)
	libggml-rpc.so => /home/deck/llamacpp/vulkan/libggml-rpc.so (0x00007f5278b0c000)
	libggml-vulkan.so => /home/deck/llamacpp/vulkan/libggml-vulkan.so (0x00007f5276c00000)
	libgomp.so.1 => /usr/lib/libgomp.so.1 (0x00007f5278ab9000)
	libvulkan.so.1 => /usr/lib/libvulkan.so.1 (0x00007f5276b7d000)

(deck@steamdeck vulkan)$ ./llama-cli --list-devices
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Graphics (RADV VANGOGH) (radv) | uma: 1 | fp16: 1 | warp size: 32 | shared memory: 65536 | int dot: 0 | matrix cores: none
Available devices:
  Vulkan0: AMD Radeon Graphics (RADV VANGOGH) (6101 MiB, 6101 MiB free)