We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hi there,
Currently I'm using cortex.tensorrt-llm in order to benchmark for MMLU and TruthfulQA using https://github.com/EleutherAI/lm-evaluation-harness/tree/main
However, the current API does not return logits_prob
logits_prob
Please add logits_prob in chat_completion API as it's supported from 0.8.0 https://github.com/NVIDIA/TensorRT-LLM/issues/983
chat_completion
https://github.com/NVIDIA/TensorRT-LLM/issues/983
Thank you
The text was updated successfully, but these errors were encountered:
Closing all open Tensorrt-llm stories due to TensorRT-LLM not supporting Desktop Parent issue: menloresearch/cortex.cpp#1742
Sorry, something went wrong.
No branches or pull requests
Hi there,
Currently I'm using cortex.tensorrt-llm in order to benchmark for MMLU and TruthfulQA using https://github.com/EleutherAI/lm-evaluation-harness/tree/main
However, the current API does not return
logits_prob
Please add
logits_prob
inchat_completion
API as it's supported from 0.8.0https://github.com/NVIDIA/TensorRT-LLM/issues/983
Thank you
The text was updated successfully, but these errors were encountered: