cuda : add f32 to bf16 copy op

[CISC]

This allows BF16 KV-cache on CUDA.

Full disclosure: I noticed this in ik_llama.cpp repo, but this is not an upstream, it was a simple feature to add.

Originally submitted as ggml-org/ggml#1182 but moved here to avoid sync conflicts.

[JohannesGaessler]

Did you look at any IK code to figure out how to do the implementation or did you just see that the repository has the feature and then figure out how to do the implementation completely on your own?

[CISC]

Did you look at any IK code to figure out how to do the implementation or did you just see that the repository has the feature and then figure out how to do the implementation completely on your own?

I quite likely looked at it at some point, as I went through all the PRs, but since I now knew this might be an issue I decided to do this by simply trying -ctk bf16 -ctv bf16, which threw an OP_CPY error, it turned out to be rather simple from there.

[CISC]

What is obviously missing here BTW is f16 to bf16 copy, but I don't know what/if that is ever triggered?

[jukofyork]

Have you noticed any benefit to using a BF16 KV-cache? I wanted to try this before but ran into the same copy error so interested to try it now and see what difference it makes for CUDA.

[CISC]

Have you noticed any benefit to using a BF16 KV-cache? I wanted to try this before but ran into the same copy error so interested to try it now and see what difference it makes for CUDA.

I didn't run any benchmarks, but it should theoretically improve PPL compared to F16, at least on certain models. Speed-wise it can swing both ways depending on hardware I guess.