How can I extract the raw weights of each layer and also split them up into sub components? (like q_proj, input RMS normalization etc.)

[dawidlouisbekker]

I want to break up weights in such a way that they can be downloaded across multiple devices.

[dawidlouisbekker]

I just want to make sure that the quantized calculations I am doing are correct so I can start hooking up devices through cross over cables, Bluetooth and LAN for now.

[dawidlouisbekker]

Never mind I found solution
import logging
import sys
from pathlib import Path
from gguf.gguf_reader import GGUFReader

logger = logging.getLogger("reader")

sys.path.insert(0, str(Path(file).parent.parent))

def read_gguf_file(gguf_file_path):
"""
Reads and prints key-value pairs and tensor information from a GGUF file in an improved format.

Parameters:
- gguf_file_path: Path to the GGUF file.
"""

reader = GGUFReader(gguf_file_path)

# List all key-value pairs in a columnized format
print("Key-Value Pairs:") # noqa: NP100
max_key_length = max(len(key) for key in reader.fields.keys())
for key, field in reader.fields.items():
    value = field.parts[field.data[0]]
    print(f"{key:{max_key_length}} : {value}") # noqa: NP100
print("----") # noqa: NP100

# List all tensors
print("Tensors:") # noqa: NP100
tensor_info_format = "{:<30} | Shape: {:<15} | Size: {:<12} | Quantization: {}"
print(tensor_info_format.format("Tensor Name", "Shape", "Size", "Quantization")) # noqa: NP100
print("-" * 80) # noqa: NP100
for tensor in reader.tensors:
    shape_str = "x".join(map(str, tensor.shape))
    size_str = str(tensor.n_elements)
    quantization_str = tensor.tensor_type.name
    print(tensor_info_format.format(tensor.name, shape_str, size_str, quantization_str)) # noqa: NP100
for tensor in reader.tensors:
    embeddings = tensor.data
    print(embeddings)

read_gguf_file()

[afsara-ben]

this works, but are the layers in order? the HF layers look more ordered