cosmos db entra id support and fixes

[TheovanKraay]

Add support for Entra ID in the auto config for Azure Cosmos DB vector store support.
This PR also adds a few fixes/enhancements and corresponding tests:

• fix to ensure getMetadata() returns the metadata fields and values
• allow metadata field to be defined as the Cosmos DB partition key (and not just /id)
• allow choice between gateway and direct mode in autoconfig

[sobychacko]

@TheovanKraay As @markpollack commented on the other issue, we could not verify certain refactoring changes we made due to the lack of access to a proper CosmosDB environment. Apologies for that bug you mentioned. Quick question: Did you get a chance to verify all the tests including the IT's on yoru CosmosDB environment? Thanks!

[TheovanKraay]

@TheovanKraay As @markpollack commented on the other issue, we could not verify certain refactoring changes we made due to the lack of access to a proper CosmosDB environment. Apologies for that bug you mentioned. Quick question: Did you get a chance to verify all the tests including the IT's on yoru CosmosDB environment? Thanks!

For this PR I have run all the ITs and they pass (using Entra ID with appropriate level of access to the Cosmos DB resources).

[sobychacko]

@TheovanKraay How do we set up the Entra ID and the access levels? Any docs for that?
Also, I am noticing that here you removed the usage of key, but noticing that it is still present in the autoconfiguration test.. Do you know why the mismatch?

[TheovanKraay]

@TheovanKraay How do we set up the Entra ID and the access levels? Any docs for that? Also, I am noticing that here you removed the usage of key, but noticing that it is still present in the autoconfiguration test.. Do you know why the mismatch?

Documentation can be found here: https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/security/how-to-grant-data-plane-role-based-access?tabs=built-in-definition%2Ccsharp&pivots=azure-interface-cli

I've left the capability to use key based auth there in the code (since we have not strictly deprecated or retired it ) but I did not include tests or docs for it because we generally don't recommend it (and it will be removed eventually).

[sobychacko]

Merged via e723371.

[sobychacko]

@TheovanKraay The version we use for Azure cosmosdb access is a bit dated from last year (5.17.1). The latest is 5.22.0 according to this. Do you see any issues with us upgrading to the latest?