Skip to content

Add support for reasoning models and token usage display #2448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 41 commits into from
Apr 2, 2025

Conversation

mattgotteiner
Copy link
Collaborator

@mattgotteiner mattgotteiner commented Mar 24, 2025

Purpose

  • Add support for o3-mini and o1, AOAI reasoning models
  • Add token usage graph in thought process

Does this introduce a breaking change?

When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.

[ ] Yes
[X] No

Does this require changes to learn.microsoft.com docs?

This repository is referenced by this tutorial
which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial,
check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.

[ ] Yes
[X] No

Type of change

[ ] Bugfix
[ ] Feature
[X] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

Code quality checklist

See CONTRIBUTING.md for more details.

  • The current tests all pass (python -m pytest).
  • I added tests that prove my fix is effective or that my feature works
  • I ran python -m pytest --cov to verify 100% coverage of added lines
  • I ran python -m mypy to check for type errors
  • I either used the pre-commit hooks or ran ruff and black manually on my code.

@mattgotteiner mattgotteiner marked this pull request as ready for review April 2, 2025 04:57
@@ -33,6 +33,7 @@ def __init__(
query_language: str,
query_speller: str,
prompt_manager: PromptManager,
reasoning_effort: Optional[str] = None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a str enum from openai library for reasoning effort? Would it be good to use that if so?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They have a literal for it:
Optional[ChatCompletionReasoningEffort]

But its not clear we should use it for what comes from the frontend.

@@ -142,6 +148,8 @@
"Enables the Azure AI Search semantic ranker, a model that re-ranks search results based on semantic similarity to the user's query.",
"useQueryRewriting":
"Enables Azure AI Search query rewriting, a process that modifies the user's query to improve search results. Requires semantic ranker to be enabled.",
"reasoningEffort":
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please translate for other files. (Perhaps we need a script for this..but its not that often yet)

@pamelafox pamelafox changed the title [In Progress] Reasoning support Add support for reasoning models and token usage display Apr 2, 2025
@pamelafox pamelafox merged commit 6cc5d19 into Azure-Samples:main Apr 2, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants