compatibility with llama.cpp

[nivibilla]

Hey, first of all, amazing work. Thanks for building an open llama model. As the title suggests, I would like to know if this model could be compatible with llamacpp

Thanks

[codesoap]

Works flawlessly with OpenAlpaca and I strongly assume this means OpenLLaMa will work as well.

Check out ggml-org/llama.cpp#1291.

[nivibilla]

Amazing, then we can take advantage of all the tools in llama.cpp. Especially the M1 gpu inference. I get 88ms per token on guanaco 13b on my MacBook. which is quite fast considering how big the model is.