ggml: use _mm[512/256]_dpbusd[_avx]_epi32 to directly accumulate into the result register

[SongXiaoXi]

• Replaced the traditional accumulate-to-zero pattern with direct accumulation into the output register

Benchmark: q4_0 model:

platform: AMD Ryzen 9 9950X (Zen 5) (AVX512VNNI)
Before:

Final estimate: PPL = 8.0754 +/- 0.05511

llama_perf_context_print:        load time =    1175.91 ms
llama_perf_context_print: prompt eval time = 1339282.64 ms / 288768 tokens (    4.64 ms per token,   215.61 tokens per second)
llama_perf_context_print:        eval time =       0.00 ms /     1 runs   (    0.00 ms per token,      inf tokens per second)
llama_perf_context_print:       total time = 1341975.38 ms / 288769 tokens

After:

Final estimate: PPL = 8.0754 +/- 0.05511

llama_perf_context_print:        load time =    1181.06 ms
llama_perf_context_print: prompt eval time = 1184860.36 ms / 288768 tokens (    4.10 ms per token,   243.71 tokens per second)
llama_perf_context_print:        eval time =       0.00 ms /     1 runs   (    0.00 ms per token,      inf tokens per second)
llama_perf_context_print:       total time = 1187563.38 ms / 288769 tokens

This demonstrates a ~12% real-world speedup in prompt evaluation performance, with no impact on accuracy.

Please let me know if:

• Any naming conventions need to be adjusted
• Additional tests are preferred

[SongXiaoXi]

Hi @Srihari-mcw! Let me know if there’s anything I can do to help move this forward. Thanks again!

[Srihari-mcw]

Will try to test this and get back. Thanks

[Srihari-mcw]

We tested with the latest changes with llama-bench on AMD Granite Ridge 9600X and observe good performance gains with the latest changes

GCC Linux :

Q4_0_8_8 Model :

model	size	params	backend	threads	test	t/s	speedup
llama 7B Q4_0	3.56 GiB	6.74 B	CPU	6	pp 512	103.90 ± 0.15
llama 7B Q4_0	3.56 GiB	6.74 B	CPU	6	pp 512	125.90 ± 0.33	21.17%
llama 7B Q4_0	3.56 GiB	6.74 B	CPU	6	tg 128	15.07 ± 0.00
llama 7B Q4_0	3.56 GiB	6.74 B	CPU	6	tg 128	15.09 ± 0.00	0.13%

GCC Version = 12.3

The machine supports the following flags by default :

system_info: n_threads = 6 (n_threads_batch = 6) / 12 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX_VNNI = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | AVX512_BF16 = 1 | LLAMAFILE = 1 | OPENMP = 1 | AARCH64_REPACK = 1 |

The PR looks good from our side. Thanks

[SongXiaoXi]

Hey, just a gentle ping. Let me know if there's anything blocking the merge!👀