When using xcframework to update llama.cpp for iOS I get out of scope errors

[unpaidcoder]

In my earlier SPM version of llama.cpp for iOS, I have llama-grammar installed.

It's now missing in the new xcframework which is giving me out of scope errors in my Swift app code.

How do I get llama-grammar installed?

Is there a toggle in the build script I should set?

https://github.com/ggml-org/llama.cpp/releases/tag/b4824

[danbev]

It's now missing in the new xcframework which is giving me out of scope errors in my Swift app code.

Could you show an example of the code that is producing this error as It should be possible to access the grammar functions that are exposed in llama.h (like llama_sampler_init_grammar, etc)?

[unpaidcoder]

Here are some of the references that throw errors. They're all related to llama_grammar functions.

[danbev]

Thanks for the references above showing the errors!

These structs and function like llama_grammar_element, are defined in the header src/llama-grammar.h which is considered a private or non public API, and they are therefore not exposed by the xcframework.

Note that the functions llama_grammar_init and llama_grammar_free where renamed in 39fbaf9 and just wanted to mention that incase you go looking for them.

But there is an API that is exposed, and the grammar related functions as well. The sampling and grammar were refactored in #9294 which might help to give some background of the changes.
These functions are available when using the xcframework, for example:

import llama
...

let grammar = """
root    ::= "1. " move " " move "\\n" ([1-9] [0-9]? ". " move " " move "\\n")+
move    ::= (pawn | nonpawn | castle) [+#]?
nonpawn ::= [NBKQR] [a-h]? [1-8]? "x"? [a-h] [1-8]
pawn    ::= ([a-h] "x")? [a-h] [1-8] ("=" [NBKQR])?
castle  ::= "O-O" "-O"?
"""
let sampler = llama_sampler_init_grammar(llama_model_get_vocab(model),
                                         grammar,
                                         "root");

if let cString = llama_sampler_name(sampler) {
    let samplerName = String(cString: cString)
    print("Sampler name: \(samplerName)")
} else {
    print("No sampler name available")
}

[unpaidcoder]

Very helpful - thank you @danbev