Choosing the Right Identity for Testing: Why I Stopped Using My Own Account

A few years ago, I was testing an Azure Function that pulled secrets from Key Vault. It worked flawlessly on my machine — every run, every time.

Confident, I pushed the changes to the build pipeline. The deployment succeeded. But a few minutes later, the production logs screamed “Authentication failed.”

I stared at the error, confused. How could something that worked perfectly in VS Code break the moment it hit the cloud?

That was the day I learned a painful truth about identity in Azure: it’s not just about whether it works — it’s about who it works for.

The Developer’s Shortcut

Like most engineers, I started with the path of least resistance.

I’d log into Azure CLI with my personal user account, run the app locally, and watch everything connect smoothly. From my perspective, that felt like “testing early.” I was validating my code and the Key Vault access logic before deploying it.

But in reality, I wasn’t testing what production would experience — I was testing what I was allowed to do.

As a user, I had Contributor access. I could read, write, and even delete the vault if I wanted. My access token was a golden key. So of course my tests passed — the system wasn’t proving my permissions, it was simply trusting them.

The False Sense of Security

In production, that same app didn’t run as me. It ran under a Managed Identity with very limited Key Vault access — just enough to read secrets, not manage them.

That subtle difference was enough to break everything. The Managed Identity wasn’t assigned the correct Key Vault role, and since my local tests ran under my overprivileged user identity, the issue never surfaced until it hit PROD.

That’s the danger of developer-led identity testing — it feels like you’re being proactive, but you’re really just validating your own access, not the system’s configuration.

The Shift to Service Principals

After that outage, I stopped using my personal account for anything beyond local tinkering. I created a Service Principal (SPN) that mimicked the production identity’s permissions — nothing more, nothing less.

It was a small change that made a huge difference. A Service Principal is like a synthetic identity. It doesn’t belong to a person; it belongs to the system. It uses a client credential flow instead of a login session, and it only has access to what you explicitly grant it.

That means when I test Key Vault access with an SPN, I’m testing exactly what my app will experience — same flow, same limitations, same RBAC boundaries.

Suddenly, my early tests became meaningful. When they passed, I knew my Managed Identity in SIT or PROD would pass too. When they failed, it was for the right reason.

The Culture of Repeatable Testing

As my team adopted this approach, something else changed: consistency.

Before, one developer’s “local success” didn’t guarantee another’s. Our CI/CD pipelines often failed because they ran under different credentials from what we used locally.

By switching all early access tests — like Key Vault, Storage, or SQL authentication — to use SPN credentials, we gained repeatability. The same pytest suite that passed on my laptop would pass on the build agent and in the SIT environment, because the identity context was identical.

It also aligned beautifully with our principle of least privilege. Developers didn’t need high-level access just to run tests — the SPN handled that, and we could scope it tightly to the resources we needed.

The Three-Stage Identity Pattern

Looking back, the lesson feels obvious, but it reshaped how I approach cloud testing:

That pattern gives you the best of all worlds: speed during development, accuracy in testing, and security in production.

Example: Testing Across Identity Types

Here’s how the same test can be adapted across those three stages using pytest and azure-identity.

1. Local Dev — Your AAD User

Run locally after az login in VS Code or CLI. Ideal for testing logic flow and secret retrieval.

# test_keyvault_user_identity.py
import os, pytest
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient

def test_keyvault_access_as_user():
    """Runs locally using your AAD user token (via az login)."""
    vault_name = os.getenv("KEY_VAULT_NAME", "kv-dev-app01")
    kv_uri = f"https://{vault_name}.vault.azure.net"
    credential = DefaultAzureCredential(exclude_managed_identity_credential=True)
    client = SecretClient(vault_url=kv_uri, credential=credential)
    secret = client.get_secret("db-connection-string")
    assert secret.value, "Secret retrieval failed under user identity"

✅ Use Case: Early dev loop
⚠️ Limitation: May mask permission issues (too much privilege)

2. CI/CD — Service Principal (SPN)

Run automatically during pipeline build/test stages. Mimics production access with fixed credentials.

# test_keyvault_spn_identity.py
import os, pytest
from azure.identity import ClientSecretCredential
from azure.keyvault.secrets import SecretClient

def test_keyvault_access_as_spn():
    """Validates Key Vault access using Service Principal credentials."""
    tenant_id = os.getenv("AZURE_TENANT_ID")
    client_id = os.getenv("AZURE_CLIENT_ID")
    client_secret = os.getenv("AZURE_CLIENT_SECRET")
    vault_name = os.getenv("KEY_VAULT_NAME")
    kv_uri = f"https://{vault_name}.vault.azure.net"

    credential = ClientSecretCredential(tenant_id, client_id, client_secret)
    client = SecretClient(vault_url=kv_uri, credential=credential)
    secret = client.get_secret("db-connection-string")
    assert secret.value, "SPN lacks access to required secret"

✅ Use Case: CI/CD validation, environment readiness
⚠️ Tip: Scope the SPN to least privilege

3. Runtime — Managed Identity (MI)

Run from within an Azure resource (Function, Container, or VM) to test the real identity bound to that resource.

# test_keyvault_managed_identity.py
import os, pytest
from azure.identity import ManagedIdentityCredential
from azure.keyvault.secrets import SecretClient

def test_keyvault_access_as_mi():
    """Validates Key Vault access using the resource’s Managed Identity."""
    vault_name = os.getenv("KEY_VAULT_NAME", "kv-sit-app01")
    kv_uri = f"https://{vault_name}.vault.azure.net"
    credential = ManagedIdentityCredential()
    client = SecretClient(vault_url=kv_uri, credential=credential)
    secret = client.get_secret("db-connection-string")
    assert secret.value, "Managed Identity missing permission to Key Vault"

✅ Use Case: Integration / SIT
⚠️ Requirement: Must run inside Azure with MI enabled

Why This Test Matters More Than It Looks

Here’s the obvious part that often goes unsaid — but it’s the real payoff of all this testing.

If your test successfully retrieves a secret like db-connection-string, it’s not just proving that Key Vault access works. It’s also confirming that:

1. The secret exists and is named consistently across environments.
2. The identity has the right permissions to read it.
3. The value inside (for example, your SQL or Storage connection string) is present and ready to be used — meaning the app will be able to reach its target resource once deployed.

That last point is subtle but critical: when Key Vault returns a connection string successfully, your app can authenticate to the downstream system without touching a single line of code.

In other words, your test doesn’t just validate secrets — it validates connectivity, configuration, and permission boundaries all at once.

But there’s an important boundary here — and it’s worth making explicit.

The Core Principle

If your Key Vault test successfully retrieves a secret, that confirms authentication and authorization to Key Vault — not necessarily validity of the secret’s contents.

In other words:

✅ You’ve proven the identity (SPN/MI/user) can access Key Vault and read the secret.
❌ You haven’t yet proven that the secret value points to a reachable or correctly-permissioned downstream resource.

That second part is intentionally out of scope for this kind of early identity test. Those deeper checks — validating that the connection string actually connects to SQL, or that a read-only role can’t perform writes — belong to a different testing layer: the integration and security QA stages.

This keeps the boundaries clean:

• Identity tests → confirm the right entity can authenticate to Key Vault and retrieve what it needs.
• Integration tests → confirm the retrieved secret works when connecting to the target resource.
• Security QA tests → confirm role-based restrictions behave as intended.

That separation means your early “shift-left” tests stay fast, safe, and focused — while downstream testing covers the rest of the trust chain.

Lessons Learned

1. If it works under your user account, that proves nothing. You’re testing your own permissions, not your system’s configuration.
2. Use SPNs for all automated and environment tests. They replicate the production flow and catch misconfigured RBAC early.
3. Treat identity testing as part of “shift left” security. Access issues are easy to fix in SIT — and painful in PROD.
4. Automate SPN-based tests. Integrate them into your pipeline so every deployment validates access before the code goes live.
5. Successful secret retrieval = validated downstream connectivity (up to the Key Vault boundary). Connection and role-based permission testing belong further down the pipeline — owned by integration and security QA.

Closing Thoughts

The irony is that the closer I tried to “test like production,” the more I relied on my own identity — until I realised that was the problem. Identity testing isn’t about you. It’s about replicating who your system really is.

So if your Key Vault, Storage, or database tests are still running under your account, enjoy that comfort while it lasts. Because one day, when PROD doesn’t trust you anymore, it’s not going to be your code that fails — it’s your assumptions.