[Feature]: Add support for attention score output

[WoutDeRijck]

🚀 The feature, motivation and pitch

Problem

vLLM currently doesn't provide access to attention scores during inference, which are essential for model analysis and interpretability research. #11862

Feature Request

Add the ability to retrieve attention scores during model inference, similar to HuggingFace's output_attentions=True parameter.

Motivation

Need to analyze token-level relationships in model outputs
Required for building visualization tools and debugging model behavior
Critical for research into attention mechanisms

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Dineshkumar-Anandan-ZS0367]

You are asking output_attentions=True or return_cross_attentions=True for getting coordinates right.

These only given by vision encoder decoder models or cross encoder models.

Which model?

[WoutDeRijck]

I don't mean to get coordinates. I am using Llama-3.1-8b, let's say I want to extract data out of the input context, then I need the attention scores to be able to visualize where the model is looking. (Pure text-based, no vision)

These are ofcourse also present in decoder-only models.

[Dineshkumar-Anandan-ZS0367]

Apologise by mistakes. I have integrated score using tensor logits already. Thanks!

[WoutDeRijck]

I do not need the logits as well. I need the attention scores.

[HuiSiqi]

Any update of this? I also need to visualize the attention scores of decoder-based models.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[oneonlee]

Any update on this?