Naming convention of operator in etdump

[yujiaoliang]

After executing a quantized model, the operator name recorded by etdump is 'Fully Connected (NC, QD8, F32, QB4W) GEMM #1'. Could you please tell me which kernel this corresponds to and how to find it?

[GregoryComer]

Are you looking for the actual kernel implementation (in code)? Or the corresponding op in the original graph?

If the question is in regards to the kernel implementation, the kernel code lives in the XNNPACK library. The naming convention for XNNPACK ops in this case is (DimOrder, Activation DType, Output Activation DType, and Weight DType). QD8 means dynamically quantized activations, 8-bit. Output activation dtype is f32. And weights are 4-bit blockwise (groupwise) quantized.

The actual kernels in XNNPACK live under https://github.com/google/XNNPACK/tree/master/src/qd8-f32-qb4w-gemm. Which specific qb4w kernel depends on your hardware at runtime, with the kernel selection logic living in https://github.com/google/XNNPACK/blob/master/src/configs/gemm-config.c#L2334.

[yujiaoliang]

Thank you for your reply. I am trying to find the actual kernel that is being used, and then replace the gemm computation within it with my custom gemm library. There might be a better approach, but I'm not sure. Anyway, thank you very much！