Improve schema of CosmosDB chat history to handle long conversations

[pamelafox]

Purpose

The original approach to CosmosDB chat history was to store both the session information and all the chat messages inside the same document. However, we received feedback from the CosmosDB team that such an architecture is risky because a single session could have so many messages that it exceeds the 2MB document size limit. The safer approach is to store the sessions and messages separately, in one document each.

This PR migrates to that schema, and is based off the same schema used in this Cosmos DB sample:
https://github.com/AzureCosmosDB/cosmosdb-nosql-copilot

It should be just as performant, as we're using hierarchical partition keys (user_id/session_id) for all the operations.

However, this PR is a breaking change: if a developer already had this deployed with history and deployed the new version, users would lose their current history and start over (in a new container). If this is an issue for some folks, we could write a migration script to migrate over the data from the old container to the new one. I'm not sure how many folks are using history in production yet, since it's a fairly new feature.

Does this introduce a breaking change?

When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.

[X] Yes
[ ] No

Does this require changes to learn.microsoft.com docs?

This repository is referenced by this tutorial
which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial,
check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.

[ ] Yes
[X] No

Type of change

[X] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[X] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

Code quality checklist

See CONTRIBUTING.md for more details.

• The current tests all pass (python -m pytest).
• I added tests that prove my fix is effective or that my feature works
• I ran python -m pytest --cov to verify 100% coverage of added lines
• I ran python -m mypy to check for type errors
• I either used the pre-commit hooks or ran ruff and black manually on my code.

[Copilot]

Copilot reviewed 5 out of 14 changed files in this pull request and generated no comments.

Files not reviewed (9)

• app/backend/requirements.txt: Language not supported
• infra/main.bicep: Language not supported
• tests/snapshots/test_cosmosdb/test_chathistory_getitem/auth_public_documents_client0/result.json: Language not supported
• tests/snapshots/test_cosmosdb/test_chathistory_query/auth_public_documents_client0/result.json: Language not supported
• tests/snapshots/test_cosmosdb/test_chathistory_query_continuation/auth_public_documents_client0/result.json: Language not supported
• .pre-commit-config.yaml: Evaluated as low risk
• app/backend/chat_history/cosmosdb.py: Evaluated as low risk
• app/backend/config.py: Evaluated as low risk
• app/frontend/src/api/api.ts: Evaluated as low risk

[graememel-kap]

This is a great change, any chance you can publish a migration script? We've been running this is production for the last few months and have roughly 3000 chat history items in our cosmos container.

[pamelafox]

@graememel-kap Sure, we can work on one this week, thanks for letting us know you need it.

[markjbrown]

@pamelafox let me know if you have questions on this. The approach I generally recommend is to create an Azure Function with a Cosmos Trigger and Output binding and set the start time to beginning and have it iterate through the collection. You can also create a simple Jupyter Notebook or python script and implement Change Feed Processor natively too.

For in-place updates you need a way to prevent the trigger from reentry. This sample shows the Azure Function route and how to prevent reentry. https://github.com/AzureCosmosDB/cosmos-embeddings-generator

[pamelafox]

We opted to write a simple script (and test), in this PR:
#2442
We're avoiding adding a function since this should only need to be run 1-2 times, and adding a function to the azd template would add a lot of complexity.
@graememel-kap Let me know if this meets your needs.

[graememel-kap]

@pamelafox Thanks! We'll test this out next week in DEV.

[TB-Shyam]

@graememel-kap Just wanted to check if the test went well when using the migration script. Also if you could provide some guide on how to run this migration script. Thank you