Is Whisper any good for multi-lingual short command sentences or could it be better?

[StuartIanNaylor]

A question as much as a statement as if you use certain languages, with short context on smaller models the WER rockets.
I have a preference for smaller models and race-till-idle of a central ASR not the consumer device kind and being central the model.
When it comes to Whisper my preference is elsewhere are similar accuracy when doing domain n-gram based LM.
WeNet did a good write up and haven't actually used yet, but its a really good comparison as training and plugging the language / domain holes is much easier and you end up with much faster models that could even have better domain WER.
n-gram based LM and https://wenet.org.cn/wenet/context.html

I am spammed the above deliberately , to ask are there any similar methods with Whisper? Or is a matter of finetuning?
@ggerganov I thought I would ask, do you have any code plans for training small domain specific models.
I have been thinking a cluster of domain specific models, than bigger all-ones, but either, but wondering if training LLMs and Langchain/Agent type mechanisms could have similiar optimisations?
I have hunch this should be possible to train a domain specific small LLM on the output of the ASR to be used as a hybrid pair.
Also it output would likely be a Domain REST api of some form of json.

I thought I would ask as much dev seems larger all-in-one LLMs or very large domains such as coding and not seen. nor know if small domain efficient training is possible or what would be a goodbase model for finetuning with context injection to train to an api, where as above similar LM and context biasing can be implemented?

[ggerganov]

#1229 might be helpful to some extend when the domain is super narrow and specific.

As for training / finetuning - maybe in the future, but at least few months down the road.

[StuartIanNaylor]

It may but have been really interested in how narrow and small a model you can make.
I was hoping you could present a model document for zones with their entities and the entities command functions as an single language english api, with the translated input lang words paired with them.
I was really interested to feed a LLM training data of the ASR to be coupled with that LLM and become its own mini hybrid whisper.
I already looked at https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md which likely would easily do the json. Wasn't sure about the llm training to expect a zone/entity/contol flow structure find the paired lang words and spit out the corresponding api 'english' json/yaml.
Maybe it could or at least you could grab a model and finetune it to do so, but started to get curious of how small a purpose trained LLM could be for the task.
I have got really curious now as all dev seems to be with larger realtively complex models that do everything and I don't think I have seen anything for narrow domains and wondered if there is a watershed moment with LLM's that need to be of x parameters before being useable?

I wondered if you where doing any dev and thought an intended use and the curiosity of how small a LLM can be in params to actually work... ? As I dunno :)

[StuartIanNaylor]

https://github.com/jmorganca/ollama
Just so a youtube so may have a go just reopened for attention as looks quite good