Can't convert llama 2 ggmlv3 to gguf

[guranu]

Hello, so i downloaded llama 2 uncensored ggml version by TheBloke but when i try to convert it to gguf i can't:

~/llama.cpp $ python convert-llama-ggmlv3-to-gguf.py --input llama2_7b_chat_uncensored.ggmlv3.q5_K_S.bin --output llama2_7b_chat_uncensored.ggmlv3.q5_K_S.gguf
* Using config: Namespace(input=PosixPath('llama2_7b_chat_uncensored.ggmlv3.q5_K_S.bin'), output=PosixPath('llama2_7b_chat_uncensored.ggmlv3.q5_K_S.gguf'), name=None, desc=None, gqa=1, eps='5.0e-06', context_length=2048, model_metadata_dir=None, vocab_dir=None, vocabtype='spm')

=== WARNING === Be aware that this conversion script is best-effort. Use a native GGUF model if possible. === WARNING ===

* Scanning GGML input file
* GGML model hyperparameters: <Hyperparameters: n_vocab=32000, n_embd=4096, n_mult=256, n_head=32, n_layer=32, n_rot=128, n_ff=11008, ftype=16>

=== WARNING === Special tokens may not be converted correctly. Use --model-metadata-dir if possible === WARNING ===

* Preparing to save GGUF file
* Adding model parameters and KV items
* Adding 32000 vocab item(s)
* Adding 291 tensor(s)
Traceback (most recent call last):
  File "/data/data/com.termux/files/home/llama.cpp/convert-llama-ggmlv3-to-gguf.py", line 345, in <module>
    main()
  File "/data/data/com.termux/files/home/llama.cpp/convert-llama-ggmlv3-to-gguf.py", line 341, in main
    converter.save()
  File "/data/data/com.termux/files/home/llama.cpp/convert-llama-ggmlv3-to-gguf.py", line 165, in save
    self.add_tensors(gguf_writer)
  File "/data/data/com.termux/files/home/llama.cpp/convert-llama-ggmlv3-to-gguf.py", line 273, in add_tensors
    mapped_name = nm.get(name)
                  ^^^^^^
AttributeError: 'TensorNameMap' object has no attribute 'get'

[KerfuffleV2]

I think your gguf version is out of sync with the convert script and/or your convert script (and llama.cpp stuff) is out of date. The line the error occurred on isn't in the current version.

[pfeiffee]

I can confirm that this occurs on the latest of everything, as of this post. Not sure what is wrong with gguf 0.3.0+

Run:
pip install gguf==0.2.1
To restore gguf to a working version.

[KerfuffleV2]

@pfeiffee I'm afraid you're mistaken. You're definitely not on the latest of everything. Here's those lines from master:

https://github.com/ggerganov/llama.cpp/blob/528134dd0267838d9c0250cf1d9621631dff09b2/convert-llama-ggmlv3-to-gguf.py#L272-L286

The code you and OP are talking about was changed by a pull that got merged two days ago: dc07dc4

The like I said, the problem is probably because your GGUF is out of sync with the script version.

[ghost]

I was getting AttributeError while trying to convert llama2uncensored, then I updated llama.cpp to 1134 (528134d), and it worked:

* Successful completion. Output saved to: /data/data/com.termux/files/home/llama2_7b_chat_uncensored.gguf.q4_0.bin

[KerfuffleV2]

It was probably #2916 or #2927 that fixed your issue. I think #2927 is going to reduce confusion a lot, at least for people running scripts out of the repo since until now it was really easy to get out of sync. If the gguf Python package version is too old or too new for the version of the script, you're going to have a bad time.

[ghost]

Yeah, almost certainly #2927 is the fix.

Every1 would be scrounging for models like fiends if not for converting, so this is a smooth transition & no big deal for me.

[KerfuffleV2]

Glad it's useful! Just keep in mind if you don't use the original model metadata when converting you're probably going to see noticeable quality differences (I think there were also some small changes to the k-quants strategy for 70B models which could also make a bit of a difference).

[ghost]

Yup, you're singing to the quire, brother. I'm loyal to the metadata:

Edit: cringe I hope OP decides to add metadata

[guranu]

Yeah, almost certainly #2927 is the fix.

Every1 would be scrounging for models like fiends if not for converting, so this is a smooth transition & no big deal for me.

How do i update it?

[guranu]

Nvm rm -rf'd and reinstalled thr repo now it converted succesfully.