vLLM is returning the prompt with the result

[Stealthwriter]

How to prevent vLLM from returning the prompt back with the result?

[teraktor2006]

If you use api_server then in vllm.entrypoints.api_server.py remove prompt from

text_outputs = [ prompt + output.text for output in request_output.outputs ]

[viktor-ferenczi]

It should be controlled by a flag in the request.

[Stealthwriter]

It should be controlled by a flag in the request.

I added echo = false while sending the request, but I got an error. I think echo is not implemented?

[Stealthwriter]

If you use api_server then in vllm.entrypoints.api_server.py remove prompt from

text_outputs = [ prompt + output.text for output in request_output.outputs ]

I will try this

[danilopeixoto]

@viktor-ferenczi Is it implemented? Or can we prepare a PR?

[PeterXiaTian]

这个问题解决了吗？我现在也碰到这个问题了，在requests发请求的时候加参数提示没有这个参数

[PeterXiaTian]

If you use api_server then in vllm.entrypoints.api_server.py remove prompt from

text_outputs = [ prompt + output.text for output in request_output.outputs ]

这个是加上还是去掉prompt?