PROTOTYPE Enable Managed Service Identity tokens obtained by Azure.Identity via IMDS to store be stored in the MSAL Cache

[henrik-me]

Issue/Feature:
Azure.Identity obtains MSI tokens via the IMDS endpoint today. In order to leverage the same caching across all libraries it may be benefitial for MSAL to allow Azure.Identity to cache MSI tokens using the MSAL cache. An alternative is to move the logic of obtaining the MSI token to MSAL.

Suggestion:
Provide an API for ConfidentialClientApplications which can take an AT and persist it in the cache. The caller will need to acquireToken silently before trying to get a token them selves and they will also need to call the persist AT API when they get a new token.

Alternative:
Move MSI token acquisition via the IMDS endpoint to MSAL and let MSAL handle this scenario.

[bgavrilMS]

We would need the following to inject an app token into the MSAL cache.

• environment and tenantid
• scope
• authentication scheme (we'd only support Bearer)
• expires in (in seconds)
• refresh in (in seconds) - optional

The API name must convey the fact that it is for APP only. aka.ms links

[bgavrilMS]

@schaabs - please have a look and we can prioritize this if Azure SDK can provide the info above.

[bgavrilMS]

As discussed with @schaabs , we should prototype this first and share it with Azure SDK via private nuget package.

[karavar]

What problem are we trying to solve here?

[henrik-me]

What problem are we trying to solve here?

Having a way for customers to cache tokens using MSAL even when the token is not obtained by MSAL. This work is specifically focused on MSI scenarios where the token is obtained by Azure.Identity.

[karavar]

Is it not possible to add the caching mechanism in Azure.Identity directly? Just curious why are we expecting developers to use two different libraries?

[jmprieur]

@karavar : Azure.Identity leverages MSAL.NET ...
As @bgavrilMS mentioned earlier, we are also looking if we want MSAL.s to support managed identity directly

[karavar]

Would that mean MSAL would be part of Azure SDK somehow? Just trying to understand what the end goal is here and how would/should a developer write code to connect to Key Vault or other target services using an MSI.

[jmprieur]

@karavar : yes, the Azure SDK takes a dependency on MSAL anyway. This issue is to investigate if we want other token sources that eSTS (in this case the MSI endpoint) to benefit from MSAL.NET's cache, and how this would work. It's purely internal, it doesn't change the current developer experience which is still using the DefaultAzureCredential if you want to use the Azure SDK directly or the DefaultCertificateLoader in (which leverages the DefaultAzureCredential) in web apps, web APIs, and daemon applications. If you are on ASP.net core, and use full Microsoft.identity.web capabilities, you just describe the certs you want: See Using certificates with Microsoft.Identity.Web

Feel free to ping me directly, if you want details.

[karavar]

Awesome! No change to developer experience for using Azure.Identity but automatic benefit of token caching. Please make that super clear in the description.

Now most other tokens are valid for much shorter period, while MSI tokens are valid for 24 hours. In addition, IMDS and ESTS-R already cache tokens. This already confused developers because the changes to authorization don’t get reflected automatically due to these caches.

If we’re adding another client side later, let’s make sure we make it clear that a cached token was retire and provide ways to flush the cache. We have not designed any customer facing way to flush the cache on IMDS and ESTS-R.

[bgavrilMS]

@karavar - can you please give more details on the authorization and caching issues described here? I'd expect IMDS to give a token for resource R, authority A (env + tenant) with expiry EXP. This should be enough for MSAL to cache the token. When there is <5 min of expiry left from the token, MSAL will attempt to fetch a new token, which we haven't really defined in the scope of MSI.

If authorization to the resource is revoked, there's nothing that MSAL can do about this (after all, it's a bearer token). The resource can reject the token if they implement CAE for example.

[karavar]

If the authorization is not based on information in the token, then everything works fine. But there are scenarios where authorization is based on the information in the token - such as group membership and AAD App Roles. That is, a managed identity is added to a group and the group is authorized via Azure RBAC for example, to have access to resources. The token includes the Group membership claim that is used by RBAC check-access APIs to grant permission. Similar story for App Roles, where an AAD App-Role is assigned to a managed identity is reflected in the MI token.
Now, when the group membership or app role configuration changes, for example, if the MI is added to a new group or removed from an existing group, the existing token must be discarded and a new token must be issued. But that does not happen for the validity of the token which is 24 hours.

[henrik-me]

If the authorization is not based on information in the token, then everything works fine. But there are scenarios where authorization is based on the information in the token - such as group membership and AAD App Roles. That is, a managed identity is added to a group and the group is authorized via Azure RBAC for example, to have access to resources. The token includes the Group membership claim that is used by RBAC check-access APIs to grant permission. Similar story for App Roles, where an AAD App-Role is assigned to a managed identity is reflected in the MI token.
Now, when the group membership or app role configuration changes, for example, if the MI is added to a new group or removed from an existing group, the existing token must be discarded and a new token must be issued. But that does not happen for the validity of the token which is 24 hours.

Such scenarios should be rejected by the resource via CAE/PAS/RBAC/ABAC not by MSAL validating tokens. The resource should send an appropriate www-authenticate header back indicating that a refresh of the token is required. MSAL have other scenarios which includes long lived tokens today. It the same for those scenarios. Perhaps what you are saying here is that IMDS is the real cache and we should only return the token from the MSAL cache in case IMDS can't be reached? (different feature but perhaps that is the one which is the wanted feature)

[karavar]

Yes, we're investigating a CAE based solution. All I am pointing out is today there is no caching in Azure.Identity. But if we start caching the 24-hour MSI tokens in Azure.Identity (via MSAL or otherwise), and there is a CAE event that invalidates IMDS cache and/or ESTS-R cache, then somehow Azure.Identity cache also needs to be invalidated. I am not sure how that'd work, but we need to think through the end to end implications of adding this cache. The IMDS and ESTS-R cache already causes a lot of confusion, we'd be adding on top of it, so let's be mindful.

[henrik-me]

Ack, that scenario is relatively well known to MSAL (for user tokens) App tokens seems to be a bit in the future for CAE.

[karavar]

I wonder if it's possible to incorporate some kind of (retry?) logic in the Azure SDK clients (like KeyVaultClient) that sends a signal to Azure.Identity/MSAL to invalidate this proposed MSAL cache, so that developers don't have to worry about having to manually flush the cache and they're not left wondering why is my code not getting the latest token.

[karavar]

Also note that CAE story for Managed Identities is not too far out. We're starting to discuss that already with the CAE team who is building an MVP story for CAE for SPs.

[maliksahil]

@karavar & @jiasli raised the concerns well. There is one other thing I would like to raise. MSI and distributed cache are fundamentally incongruent, and implemented as is will lead to bad patterns from a developer point of view when using MSI. We should keep that in mind when we implement caching with MSI - perhaps MSI caching should be somehow limited to just in memory cache?

[karavar]

@jiasli Your issue seems to be related to this prototype but deserves its own tracking item. Can you please create one? I feel we should not overload this prototyping effort of using MSAL for caching without changing any developer experiences of Azure.Identity.

[rayluo]

@jiasli Your issue seems to be related to this prototype but deserves its own tracking item. Can you please create one? I feel we should not overload this prototyping effort of using MSAL for caching without changing any developer experiences of Azure.Identity.

Indeed. I would suggest @jiasli to copy his comment to MSAL Python's issue 58. We will reopen that issue and keep track there.

[bgavrilMS]

Closing this as a different path was chosen.