Azure.RequestFailedException in Azure.Security.KeyVault.Secrets

Question

I have developed a service running in an Azure Virtual Machine Scale Set that accesses configuration from the Azure Key Vault. Recently, when debugging this service, I have started to receive a RequestFailedException when accessing the configuration.

In Azure, the Key Vault is accessed through a User-Assigned Managed Identity that has the appropriate permissions. The service is currently working fine in Azure, but I am unable to debug locally. I have selected my Azure account in the Azure Service Authentication configuration section of Visual Studio 2019 version 16.8.6, and this account has permissions to the Key Vault (which is configured to allow access from all networks).

Here is a snippet showing how I access the key vault using Azure Identity 1.2.2 and Azure.Security.KeyVault.Secrets 4.1.0 on the .NET Core 3.1 platform:

private const string KeyVault = "https://<key vault name>.vault.azure.net/";

var client = new SecretClient(new Uri(KeyVault), new DefaultAzureCredential());
var secret = client.GetSecret("<name of secret in key vault>");

Apparently the DefaultAzureCredential uses the VisualStudioCredential to resolve access, but this has somehow been broken by an update to Visual Studio (the only thing changed since my last successful debug session) or possibly a change in policy from Azure Key Vault.

Full exception

Exception thrown: 'Azure.RequestFailedException' in Azure.Security.KeyVault.Secrets.dll
An unhandled exception of type 'Azure.RequestFailedException' occurred in Azure.Security.KeyVault.Secrets.dll
Service request failed.
Status: 403 (Forbidden)

Content:
{"error":{"code":"Forbidden","message":"Access denied to first party service.\r\nCaller: name=from-infra;tid=<redacted>;appid=<redacted>;iss=https://sts.windows.net/<appid>/\r\nVault: <key vault name>;location=australiaeast","innererror":{"code":"AccessDenied"}}}

Upgrading the library

I upgraded the Azure.Identity library to version 1.3.0. This produced a more detailed exception:

Microsoft.Extensions.Configuration.AzureAppConfiguration.KeyVaultReferenceException: SharedTokenCacheCredential authentication failed: AADSTS9002332: Application 'cfa8b339-82a2-471a-a3c9-0fc0be7a4093'(Azure Key Vault) is configured for use by Azure Active Directory users only.

The full exception is described in this question. It seems the DefaultAzureCredential is attempting to use a SharedTokenCacheCredential. I updated my code to explicitly disable shared token cache credentials:

private const string KeyVault = "https://<key vault name>.vault.azure.net/";

var options = new DefaultAzureCredentialOptions { ExcludeSharedTokenCacheCredential = true };
var client = new SecretClient(new Uri(KeyVault), new DefaultAzureCredential(options));

However this produces the original exception as described above. The accepted answer to the linked question encodes the tenant id, client id, and user secret into the application which I am reluctant to do. Example documentation uses the alternate libraries Microsoft.Azure.Services.AppAuthentication and Microsoft.Azure.KeyVault.

My service is running fine in Azure. I would like to debug it locally, as I was a couple of months ago. Is there anything I can do?

Answer 1

To work around this issue without hard-coding secrets, disable the Visual Studio authentication credential and use an alternate authentication method.

In my case, I used the AzureCliCredential. First I logged into Azure in my development environment with the account that has access to the Key Vault (the same account selected in Visual Studio's 'Azure Service Authentication' configuration):

az login

Then I updated the code to either use the DefaultAzureCredential with the VisualStudioCredential disabled, or used a ChainedTokenCredential selecting the appropriate methods:

var options = new DefaultAzureCredentialOptions { ExcludeVisualStudioCredential = true };
var client = new SecretClient(new Uri(KeyVault), new DefaultAzureCredential(options));

Or:

var credentials = new ChainedTokenCredential(new ManagedIdentityCredential(...), new AzureCliCredential());
var client = new SecretClient(new Uri(KeyVault), credentials);

I believe this is a bug with Visual Studio and an issue has been logged.