Skip to content

Commit 885e5a8

Browse files
rgerganoviThalay
authored andcommitted
ggml : add RPC backend (llama/6829)
* ggml : add RPC backend The RPC backend proxies all operations to a remote server which runs a regular backend (CPU, CUDA, Metal, etc). * set TCP_NODELAY * add CI workflows * Address review comments * fix warning * implement llama_max_devices() for RPC * Address review comments * Address review comments * wrap sockfd into a struct * implement get_alignment and get_max_size * add get_device_memory * fix warning * win32 support * add README * readme : trim trailing whitespace * Address review comments * win32 fix * Address review comments * fix compile warnings on macos
1 parent 4ea3e31 commit 885e5a8

File tree

2 files changed

+1047
-0
lines changed

2 files changed

+1047
-0
lines changed

0 commit comments

Comments
 (0)