-
-
Notifications
You must be signed in to change notification settings - Fork 7.8k
[Feature]: Apply tool calling after reasoning steps in Reasoning models. #14490
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I think this restriction can be removed, and currently DeepSeek does not support function calls. It would be quite straightforward to implement for QwQ-32B. |
I've finished a version. After testing, I'll submit a PR. |
This issue is duplicated with #14429 |
Thank you for your reminder. Indeed, there are some repetitions, but I mentioned two points in the feature. Besides making vllm able to use tool calling and reasoning at the same time (this part you mentioned has been covered in #14429, and I will add the citation), I also hope to support tool calling after the reasoning steps. I think these two points are distinct. For example, previously vllm could support both reasoning and structured output simultaneously, but it did not support structured output after the reasoning steps (#12619). This issue has just been fixed recently (#12955, #14114). |
Thanks for your contribution! I'm so happy to see this issue resolved. |
Uh oh!
There was an error while loading. Please reload this page.
🚀 The feature, motivation and pitch
QwQ-32B has just been open-sourced as a reasoning model that also supports tool calling. But right now, vLLM can't support tool calling and reasoning at the same time (#14429), nor can it support tool calling after reasoning.
If we could enable the reasoning model to perform tool calling after reasoning steps, it might lead to improved performance.
Alternatives
No response
Additional context
No response
Before submitting a new issue...
The text was updated successfully, but these errors were encountered: