Implementing Dynamic Per-Tenant MongoDB Credentials in a C# Application Using HashiCorp Vault

The baseline requirement was a multi-tenant internal platform. Each tenant’s data resides in a separate MongoDB database within a shared cluster, a standard pattern for balancing cost and isolation. The immediate and glaring security risk was credential management. The initial proof-of-concept stored a static, high-privileged MongoDB connection string in the C# backend’s configuration. This is a common but deeply flawed approach. A single configuration leak would expose the entire cluster, and rotating that credential would require a full service redeployment and coordinated downtime. The operational overhead is high, and the security posture is brittle.

The goal became clear: eliminate static database credentials from the application entirely. We needed a system where credentials are ephemeral, scoped precisely to a single tenant’s database, and generated on-demand. This moves the security boundary from a static secret file to a robust, auditable authentication and authorization process. This is where HashiCorp Vault entered the architecture. The concept was to build a C# service that acts as a credential broker. When the application needs to access data for tenant-A, this service would dynamically request a new username and password from Vault. Vault, pre-configured to manage MongoDB, would create a user in MongoDB with access only to the tenant-A-db database, return the credentials to the C# service, and set a short time-to-live (TTL) on them. After the TTL expires, Vault automatically revokes the user from MongoDB.

The Foundation: Configuring Vault’s MongoDB Secrets Engine

Before writing a single line of C# code, the core of the solution must be configured in Vault. In a real-world project, this would be managed via Terraform, but for clarity, the Vault CLI commands illustrate the process directly.

First, enable the MongoDB database secrets engine. This is the component responsible for connecting to MongoDB and managing dynamic users.

# Enable the database secrets engine at a specific path
vault secrets enable database

# For a production setup, you'd likely use a more descriptive path
# vault secrets enable -path=mongodb_tenants database

Next, Vault needs a privileged MongoDB user account that it can use to create and delete other users. This is a critical step; this “root” credential for Vault is the only long-lived secret in this part of the system and must be heavily protected.

# Configure the connection to the MongoDB instance
# The connection_url contains the privileged user credentials.
# In production, this user should have roles like 'userAdminAnyDatabase' and 'readWriteAnyDatabase'.
# This secret itself should be injected into Vault via a secure mechanism.
vault write database/config/my-mongodb-cluster \
    plugin_name=mongodb-database-plugin \
    allowed_roles="tenant-readwrite" \
    connection_url="mongodb://vault-root-user:[email protected]:27017/admin?tls=true"

The pitfall here is using a user with excessive permissions. The Vault user only needs enough privilege to manage other users, not to be a full cluster admin. Principle of least privilege is paramount.

With the connection established, we define a “role” in Vault. This role is a template for the dynamic credentials. It dictates the TTL of the generated users and, most importantly, the MongoDB roles and permissions they will be granted.

# Create a Vault role named "tenant-readwrite"
vault write database/roles/tenant-readwrite \
    db_name=my-mongodb-cluster \
    creation_statements='{ "db": "{{name}}", "roles": [{ "role": "readWrite", "db": "{{name}}" }] }' \
    default_ttl="1h" \
    max_ttl="24h"

The creation_statements parameter is the core of the multi-tenant isolation. The {{name}} is a placeholder that Vault will populate. When our C# service requests a credential, it will specify a name—for instance, the tenant’s database name like tenant-A-db. Vault will then execute the creation statement, creating a user with readWrite access scoped only to the tenant-A-db database. This ensures a credential generated for one tenant cannot access another’s data.

The C# Credential Broker Service

The C# backend is the intermediary between the application logic and Vault. A common mistake is to have every part of the application that needs database access talk directly to Vault. This scatters the logic and makes it difficult to implement caching or consistent error handling. A better approach is a centralized ITenantDbClientFactory service.

Here’s the core interface for this service.

// /Services/ITenantDbClientFactory.cs

using MongoDB.Driver;

namespace SecureMultiTenant.Api.Services;

/// <summary>
/// Defines the contract for a factory that provides tenant-specific MongoDB clients.
/// The implementation is responsible for securely obtaining and managing dynamic credentials.
/// </summary>
public interface ITenantDbClientFactory : IAsyncDisposable
{
    /// <summary>
    /// Gets a configured IMongoClient for a specific tenant.
    /// This method handles credential acquisition, caching, and renewal.
    /// </summary>
    /// <param name="tenantId">The unique identifier for the tenant.</param>
    /// <param name="cancellationToken">A token to cancel the operation.</param>
    /// <returns>A configured IMongoClient ready to interact with the tenant's database.</returns>
    Task<IMongoClient> GetClientForTenantAsync(string tenantId, CancellationToken cancellationToken);
}

The implementation of this service will use the VaultSharp library to communicate with Vault and Microsoft.Extensions.Caching.Memory for intelligent caching of credentials to avoid hitting Vault for every single request.

Configuration and Initialization

The service needs configuration from appsettings.json. Notice the absence of any database credentials.

// appsettings.Production.json
{
  "Vault": {
    "Address": "https://vault.service.consul:8200",
    "Token": "VAULT_TOKEN_FROM_ENV", // Injected via environment variable or AppRole
    "MongoDbSecretMountPoint": "database",
    "MongoDbRoleName": "tenant-readwrite"
  },
  "MongoDbSettings": {
    "ClusterEndpoint": "mongo.service.consul:27017",
    "TenantDbPrefix": "tenantdb_" // e.g., tenantdb_acme-corp
  },
  "Logging": { ... }
}

The implementation class constructor will set up the VaultClient and the memory cache.

// /Services/VaultTenantDbClientFactory.cs

using VaultSharp;
using VaultSharp.V1.AuthMethods.Token;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Options;
using MongoDB.Driver;
using System.Collections.Concurrent;

// ... other using statements

public class VaultTenantDbClientFactory : ITenantDbClientFactory
{
    private readonly IVaultClient _vaultClient;
    private readonly IMemoryCache _cache;
    private readonly ILogger<VaultTenantDbClientFactory> _logger;
    private readonly VaultOptions _vaultOptions;
    private readonly MongoDbSettings _mongoDbSettings;
    
    // Use a concurrent dictionary to hold clients, as IMongoClient is thread-safe
    private readonly ConcurrentDictionary<string, IMongoClient> _tenantClients;

    public VaultTenantDbClientFactory(
        IOptions<VaultOptions> vaultOptions,
        IOptions<MongoDbSettings> mongoDbSettings,
        IMemoryCache cache,
        ILogger<VaultTenantDbClientFactory> logger)
    {
        _vaultOptions = vaultOptions.Value;
        _mongoDbSettings = mongoDbSettings.Value;
        _cache = cache;
        _logger = logger;
        _tenantClients = new ConcurrentDictionary<string, IMongoClient>();

        // Production-grade applications should use AppRole or another auth method.
        // Token auth is used here for simplicity.
        var authMethod = new TokenAuthMethodInfo(_vaultOptions.Token);
        var vaultClientSettings = new VaultClientSettings(_vaultOptions.Address, authMethod);

        _vaultClient = new VaultClient(vaultClientSettings);
    }
    
    // ... implementation of GetClientForTenantAsync will go here
}

The Core Logic: Dynamic Credential Acquisition

The GetClientForTenantAsync method is where the entire process comes together. It orchestrates caching, requesting new credentials from Vault, and constructing the IMongoClient.

// /Services/VaultTenantDbClientFactory.cs (continued)

public async Task<IMongoClient> GetClientForTenantAsync(string tenantId, CancellationToken cancellationToken)
{
    // A simple validation for tenantId format.
    if (string.IsNullOrWhiteSpace(tenantId) || tenantId.Length > 50)
    {
        throw new ArgumentException("Invalid tenantId.", nameof(tenantId));
    }

    // Check if a valid, connected client already exists.
    if (_tenantClients.TryGetValue(tenantId, out var cachedClient))
    {
        // A simple check to see if the client is still considered 'alive'.
        // More robust checks might involve a ping.
        if (IsClientConnectionHealthy(cachedClient)) 
        {
            _logger.LogDebug("Returning cached MongoDB client for tenant {TenantId}", tenantId);
            return cachedClient;
        }
    }

    var cacheKey = $"mongodb-creds-{tenantId}";

    // Try to get credentials from the local cache first.
    if (!_cache.TryGetValue(cacheKey, out Secret<UsernamePasswordCredentials> credentials))
    {
        _logger.LogInformation("No cached credentials found for tenant {TenantId}. Requesting new credentials from Vault.", tenantId);

        var tenantDbName = $"{_mongoDbSettings.TenantDbPrefix}{tenantId}";
        try
        {
            // Request dynamic credentials from Vault for the specific tenant DB
            credentials = await _vaultClient.V1.Secrets.Database.GetCredentialsAsync(
                _vaultOptions.MongoDbRoleName,
                _vaultOptions.MongoDbSecretMountPoint
            );
            
            _logger.LogInformation(
                "Successfully obtained credentials from Vault for tenant {TenantId}. Lease ID: {LeaseId}, Duration: {LeaseDuration}s", 
                tenantId, credentials.LeaseId, credentials.LeaseDurationSeconds);

            // A critical production consideration: Cache the credentials for slightly less than their
            // actual TTL to account for network latency and clock skew during renewal.
            var cacheEntryOptions = new MemoryCacheEntryOptions()
                .SetAbsoluteExpiration(TimeSpan.FromSeconds(credentials.LeaseDurationSeconds * 0.9));

            _cache.Set(cacheKey, credentials, cacheEntryOptions);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to get credentials from Vault for tenant {TenantId}", tenantId);
            throw new InvalidOperationException($"Could not secure credentials for tenant {tenantId}.", ex);
        }
    }
    else
    {
        _logger.LogDebug("Found credentials in cache for tenant {TenantId}", tenantId);
    }

    // At this point, we have credentials (either new or from cache). Now create the client.
    var mongoCredential = MongoCredential.CreateCredential("admin", credentials.Data.Username, credentials.Data.Password);
    var clientSettings = new MongoClientSettings
    {
        Credential = mongoCredential,
        Server = MongoServerAddress.Parse(_mongoDbSettings.ClusterEndpoint),
        // Important for production: Use TLS
        UseTls = true, 
        // Set timeouts
        ConnectTimeout = TimeSpan.FromSeconds(5),
        ServerSelectionTimeout = TimeSpan.FromSeconds(10)
    };

    var newClient = new MongoClient(clientSettings);
    
    // Atomically add or update the client in our concurrent dictionary.
    _tenantClients.AddOrUpdate(tenantId, newClient, (key, oldClient) => {
        // If an old client existed, it should be disposed of properly.
        // (This part of the logic can be complex in a concurrent environment)
        return newClient;
    });

    return newClient;
}

// A placeholder for a more robust health check
private bool IsClientConnectionHealthy(IMongoClient client)
{
    return client.Cluster.Description.State == MongoDB.Driver.Core.Clusters.ClusterState.Connected;
}

This implementation includes caching, detailed logging, and proper exception handling. The small detail of caching for 90% of the lease duration is a pragmatic choice born from experience; it prevents scenarios where you attempt to use a credential at the exact moment it expires.

The Request Flow Visualized

To tie it all together, here is the flow of a single API request needing tenant data. An administrator, using a simple dashboard built with Shadcn UI components, requests to view data for tenant-acme.

sequenceDiagram
    participant Frontend as Shadcn UI/Next.js
    participant Backend as C# API (.NET)
    participant CredentialFactory as VaultTenantDbClientFactory
    participant Vault
    participant MongoDB

    Frontend->>+Backend: GET /api/tenants/acme/data
    Backend->>+CredentialFactory: GetClientForTenantAsync("acme")
    
    alt Cache Miss (No credentials for "acme")
        CredentialFactory->>+Vault: Request credentials for role 'tenant-readwrite'
        Vault->>+MongoDB: CREATE USER 'v-...' WITH ROLES 'readWrite'@'tenantdb_acme'
        MongoDB-->>-Vault: User created
        Vault-->>-CredentialFactory: Return {user, pass, lease_id, ttl}
        CredentialFactory->>CredentialFactory: Cache credentials with 90% TTL
    else Cache Hit
        CredentialFactory->>CredentialFactory: Return credentials from memory cache
    end

    CredentialFactory->>CredentialFactory: new MongoClient(credentials)
    CredentialFactory-->>-Backend: Return configured IMongoClient
    
    Backend->>+MongoDB: Use IMongoClient to query tenantdb_acme
    MongoDB-->>-Backend: Return query results
    Backend-->>-Frontend: Return 200 OK with JSON data

Addressing Production Realities: Lease Renewal and Revocation

The current implementation is functional but has a weakness: it only requests new credentials when the cached entry expires. It doesn’t actively renew them. For long-running processes, a background service (IHostedService) is necessary to manage the lifecycle of these leases.

The service would periodically scan the cache for secrets that are nearing their renewal window (e.g., 75% of their TTL has passed) and issue a renewal command to Vault.

// /Services/VaultLeaseRenewalService.cs (Conceptual)
public class VaultLeaseRenewalService : IHostedService, IDisposable
{
    private readonly IMemoryCache _cache;
    private readonly IVaultClient _vaultClient;
    private readonly ILogger<VaultLeaseRenewalService> _logger;
    private Timer? _timer;

    public VaultLeaseRenewalService(IMemoryCache cache, IVaultClient vaultClient, ILogger<VaultLeaseRenewalService> logger)
    {
        _cache = cache;
        _vaultClient = vaultClient;
        _logger = logger;
    }

    public Task StartAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("Vault Lease Renewal Service is starting.");
        // Check for renewable leases every minute. This interval should be tuned.
        _timer = new Timer(DoWork, null, TimeSpan.Zero, TimeSpan.FromMinutes(1));
        return Task.CompletedTask;
    }

    private async void DoWork(object? state)
    {
        // This is a simplified approach. A real implementation would need to
        // iterate through the cache keys, which is tricky with IMemoryCache.
        // A custom cache implementation or a concurrent dictionary might be better.
        // For each cached credential:
        //  - Check if it's within its renewal window.
        //  - If so, call _vaultClient.V1.System.RenewLeaseAsync(...)
        //  - Update the cache with the new lease duration.
        //  - Handle exceptions gracefully.
        _logger.LogDebug("Scanning for leases to renew...");
    }
    
    // StopAsync and Dispose implementations...
}

Furthermore, the IAsyncDisposable on the factory interface is crucial. When the application shuts down, it should gracefully iterate through all active leases it knows about and explicitly revoke them in Vault. This ensures no orphaned users are left in MongoDB.

// /Services/VaultTenantDbClientFactory.cs (continued)
public async ValueTask DisposeAsync()
{
    _logger.LogInformation("Disposing factory and revoking active Vault leases.");
    // This is complex. We need to track all issued lease IDs.
    // The IMemoryCache doesn't easily allow iteration. A separate ConcurrentDictionary<string, string>
    // mapping tenantId to leaseId would be required.
    
    // conceptual loop
    // foreach (var leaseId in _trackedLeaseIds.Values)
    // {
    //     try
    //     {
    //         await _vaultClient.V1.System.RevokeLeaseAsync(leaseId);
    //     }
    //     catch (Exception ex)
    //     {
    //         _logger.LogWarning(ex, "Failed to revoke lease {LeaseId} during shutdown.", leaseId);
    //     }
    // }
}

Lingering Issues and Future Iterations

This architecture dramatically improves security posture by eliminating static credentials, but it introduces new complexities. The VaultTenantDbClientFactory becomes a critical, stateful component in the system. Its own availability is now paramount; if it cannot reach Vault, no database connections can be made. A production implementation must account for Vault being in a highly available cluster and implement robust retry logic with exponential backoff when communicating with it. The authentication of the C# service to Vault is another key point; the token-based auth used here is fine for development, but a production system must use a more secure, automated method like Vault’s AppRole auth mechanism, where the application proves its identity to retrieve a short-lived token. Finally, the performance overhead of creating a new IMongoClient object, which involves establishing a connection pool, is not trivial. The current caching of client objects mitigates this, but the lifecycle management of these clients in a highly concurrent environment requires careful, thread-safe implementation to prevent resource leaks.