Output hidden states like hugging face

[Sami6720]

Hi,

I am trying to get all the hidden states from the forward pass of a llama model using vllm. Sort of like what is possible with hugging face's out_hidden_states = True but with all the optimizations that film does to speed up inference that one doesn't get using vanilla hf.

Is this even possible?

I have tried setting task = embedding in the LLM class, but that only gives the last layer's last embedding. I would want to have access to the outputs from each layer.

[DarkLight1337]

See #15434