Skip to content

A Queue-Backed Azure Container Apps SaaS Blueprint: Cosmos DB, Service Bus, and Bicep

A practical, medium-scale SaaS architecture on Azure Container Apps, with Cosmos DB, Service Bus, Bicep, delivery practices, and operational guardrails.

Ekanatha Reddy Urivakili19 min3,607 words

Context and scale

This blueprint is for a multi-tenant SaaS that accepts customer commands, persists tenant-owned records, and performs some work asynchronously: provisioning, notifications, exports, or integrations. It uses a small number of .NET services rather than decomposing every capability into a separately deployed service.

The core is an ASP.NET Core API and a background worker deployed to Azure Container Apps. Azure Service Bus separates request acceptance from asynchronous processing. Azure Cosmos DB for NoSQL stores tenant-partitioned operational data. Bicep owns the infrastructure, and a staging environment exercises the same deployment path as production.

These are planning assumptions, not measured demand or Azure capacity recommendations:

DimensionAssumption
Users25,000 daily active users across 2,000 tenant accounts
Requests50 API requests/s average; 250 requests/s short peak; 3 queued messages/s average
Data volume200 GB initially, growing by roughly 25 GB/month; tenant records and operational events, not large binary files
Team size6 engineers across one product team, with a second team possible as the product grows
Monthly budget$3,000 planning target for the production region and its shared platform costs

The system is intentionally single-region. Cosmos DB and Service Bus are configured for zone redundancy where the chosen Azure region and service tier support it. That protects against some availability-zone failures, not a regional outage. A secondary region is a later, explicit investment.

The design assumes tenant identity is established by a trusted authentication system, such as Microsoft Entra External ID or an enterprise identity provider. Application authorization still determines which tenant and records a user may access. A tenant ID supplied in an untrusted request body is never sufficient authorization.

Requirements

Functional requirementDesign response
Authenticate users and scope actions to a tenantValidate identity and tenant membership at the API boundary; authorize each operation
Accept commands and return promptlyValidate, persist the command and an outbox event, then return a resource or operation identifier
Process work asynchronouslyDispatch outbox events to Service Bus; run a separately scaled Container Apps worker
Read tenant-owned dataQuery Cosmos DB using the tenant partition key and enforce tenant scope in repository methods
Support retries and duplicate deliveryMake consumers idempotent using event IDs and tenant-scoped records
Operate and deploy safelyUse Bicep, immutable image tags, staging, automated tests, and revision-based rollout
Non-functional requirementTarget or constraint
AvailabilityTarget 99.9% monthly availability for the API, subject to dependencies and a defined SLI
LatencyAPI read/write p95 below 500 ms under agreed production load; asynchronous completion is a separate SLI
DurabilityPersist accepted commands before acknowledging them; retain failed messages for inspection and replay
IsolationTenant-scoped access, partitioning, and tests for cross-tenant access; noisy-neighbor controls are explicit
ScalingScale API on HTTP demand and worker on queue backlog; cap maximum replicas to protect downstream capacity
SecurityManaged identities for Azure service access; no account keys in application configuration
RecoveryDocumented restore and replay procedures; regional recovery is not claimed until implemented and tested

Treat latency and availability targets as product decisions to validate with real traffic. An SLO without a measurement definition and a response plan is just a number in a document.

High-level design

flowchart LR
    User["Browser or client"] --> Edge["Azure Front Door or API ingress"]
    Edge --> Api["Azure Container Apps API"]
    Api --> Cosmos["Azure Cosmos DB for NoSQL"]
    Api --> Outbox["Outbox documents in Cosmos DB"]
    Outbox --> Dispatcher["Container Apps outbox dispatcher"]
    Dispatcher --> Bus["Azure Service Bus queue"]
    Bus --> Worker["Container Apps worker"]
    Worker --> Cosmos
    Api --> Identity["Microsoft Entra ID"]
    Worker --> Monitor["Azure Monitor and Log Analytics"]
    Api --> Monitor

Caption: Commands enter through Container Apps, commit to Cosmos DB, and reach workers through Service Bus.

The API is the synchronous boundary. It authenticates the caller, checks tenant authorization, validates the command, and writes both the business record and an outbox record in a Cosmos DB transactional batch. Those documents must share a logical partition key for the batch to be atomic. The API returns only after that write succeeds.

An outbox dispatcher reads committed events and sends them to Service Bus. It may send an event more than once: the send and the dispatcher checkpoint cannot be made one atomic transaction across Cosmos DB and Service Bus. Consumers therefore deduplicate by event ID. Service Bus provides durable buffering and retry handling; it does not make application side effects exactly once.

The worker uses a managed identity to receive messages and access Cosmos DB. It commits its idempotency marker and business side effect in one Cosmos transactional batch, then completes the message. If it fails before completion, redelivery is safe. If a message repeatedly fails, it eventually reaches the dead-letter queue for investigation.

Azure Container Apps supplies ingress, revisions, scaling, and container execution. It is not a substitute for application-level timeouts, idempotency, capacity limits, or dependency health checks. Put the public edge behind Azure Front Door when the product needs a global entry point, WAF policies, or edge caching; it is optional in this regional baseline.

Cloud service mapping

ConcernAzure choiceAWS equivalentGCP equivalent
Containerized API and workerAzure Container AppsAWS App Runner or ECS on FargateCloud Run
Durable queueAzure Service BusAmazon SQSCloud Tasks or Pub/Sub, depending on semantics
Tenant operational storeAzure Cosmos DB for NoSQLAmazon DynamoDBFirestore or Bigtable, depending on access patterns
Container image registryAzure Container RegistryAmazon ECRArtifact Registry
Workload identityManaged identities and Microsoft Entra IDIAM roles for tasksWorkload Identity
Infrastructure as codeBicep and Azure Resource ManagerAWS CDK or CloudFormationTerraform or Google Cloud deployment tooling
Logs and metricsAzure Monitor and Log AnalyticsCloudWatchCloud Monitoring and Cloud Logging
Secrets, if requiredAzure Key VaultAWS Secrets ManagerSecret Manager

These are service-level analogies, not interchangeable APIs. In particular, queue ordering, delivery, deduplication, and transaction capabilities differ. Select a cloud-native service based on required semantics rather than assuming a migration between equivalents is a configuration change.

Data design

The primary Cosmos DB container uses /tenantId as its partition key. A tenant’s records can then be queried and, where needed, transactionally updated together. The model is deliberately denormalized: Cosmos DB does not provide relational joins across partitions, so read models should match the product’s access patterns.

erDiagram
    TENANT ||--o{ SUBSCRIPTION : owns
    TENANT ||--o{ ORDER : owns
    TENANT ||--o{ OUTBOX_EVENT : emits
    TENANT ||--o{ INBOX_RECORD : processes
    ORDER ||--o{ OUTBOX_EVENT : produces
    OUTBOX_EVENT ||--o| INBOX_RECORD : deduplicated_by

Caption: These logical records are tenant-scoped documents in a Cosmos DB container, not relational tables.

Document typeID conventionPartition keyPurpose
Tenanttenant:{tenantId}tenantIdTenant settings and status
Subscriptionsubscription:{subscriptionId}tenantIdPlan and billing-related application state
Orderorder:{orderId}tenantIdCustomer-owned resource and current status
Outbox eventoutbox:{eventId}tenantIdDurable event to dispatch after a business write
Inbox recordinbox:{eventId}tenantIdEvidence that a consumer applied an event

Cosmos DB for NoSQL is schemaless and does not use relational SQL DDL to create tables. The operational equivalent of the container definition belongs in Bicep: create one container with partition key /tenantId and an autoscale throughput policy sized from observed request-unit consumption. Do not copy a guessed RU/s setting between environments and call it capacity planning.

A representative order and event might look like this:

{
  "id": "order:ord_01J9A7",
  "tenantId": "ten_42",
  "type": "order",
  "status": "accepted",
  "createdAt": "2026-11-04T12:00:00Z",
  "version": 1
}
{
  "id": "outbox:evt_01J9A8",
  "tenantId": "ten_42",
  "type": "outbox",
  "eventId": "evt_01J9A8",
  "eventType": "OrderAccepted",
  "aggregateId": "ord_01J9A7",
  "occurredAt": "2026-11-04T12:00:00Z",
  "schemaVersion": 1
}

The API always includes tenantId in its query partition key and derives it from authorized identity context. A composite business identifier is not a security boundary. Establish retention rules for outbox and inbox records: deleting deduplication records too early reopens the possibility that a delayed message repeats a side effect.

Partitioning by tenant is appropriate at this scale only if tenant sizes are reasonably distributed. A single exceptionally large tenant can become a hot logical partition. Monitor per-partition behavior and plan a migration strategy before adopting a partition key that cannot be changed in place.

Low-level design

Keep the codebase modular even if the API and worker initially share a repository. Useful modules include identity and tenant access, orders, subscriptions, outbox dispatch, and message handling. Each module owns its contracts and persistence operations. Avoid a shared “common” layer that becomes a second, implicit domain model.

classDiagram
    class OrdersApi
    class TenantContext
    class OrderService
    class CosmosOrderRepository
    class OutboxDispatcher
    class ServiceBusPublisher
    class OrderWorker
    class InboxRepository
    OrdersApi --> TenantContext
    OrdersApi --> OrderService
    OrderService --> CosmosOrderRepository
    OrderService --> OutboxDispatcher
    OutboxDispatcher --> ServiceBusPublisher
    OrderWorker --> InboxRepository
    OrderWorker --> CosmosOrderRepository

Caption: Keep tenant authorization at the API boundary and idempotency inside the message-handling path.

A useful boundary is “command accepted” versus “command completed.” The API can return 202 Accepted with an operation or resource URL when work is genuinely asynchronous. Do not return success for a write that has not yet been durably recorded. Use an explicit status model such as accepted, processing, completed, and failed where users need to track progress.

Minimal example: publish a message with managed identity. Add the Azure.Messaging.ServiceBus and Azure.Identity packages to a .NET project. The environment variable contains the Service Bus fully qualified namespace, not a connection string. Locally, DefaultAzureCredential can use a developer identity; in Azure, it uses the app’s managed identity.

using Azure.Identity;
using Azure.Messaging.ServiceBus;

var namespaceName = Environment.GetEnvironmentVariable("SERVICEBUS_NAMESPACE")
    ?? throw new InvalidOperationException("SERVICEBUS_NAMESPACE is required.");

await using var client = new ServiceBusClient(
    namespaceName,
    new DefaultAzureCredential());

var sender = client.CreateSender("work");

var message = new ServiceBusMessage("""
    {"eventId":"evt-123","tenantId":"ten-42","type":"OrderAccepted"}
    """)
{
    MessageId = "evt-123",
    Subject = "OrderAccepted"
};

await sender.SendMessageAsync(message);

Notice that MessageId is stable across retries; it helps broker-side duplicate detection when enabled, but consumer idempotency is still required.

Realistic example: apply a message idempotently. This worker uses a single tenant partition and a transactional batch to create an inbox marker and update the order projection atomically. A conflict on the inbox marker means the event was already applied. In production, validate the envelope and schema version, bound message processing time, and emit useful failure context.

using System.Net;
using Azure;
using Azure.Identity;
using Azure.Messaging.ServiceBus;
using Microsoft.Azure.Cosmos;

var cosmosEndpoint = Environment.GetEnvironmentVariable("COSMOS_ENDPOINT")
    ?? throw new InvalidOperationException("COSMOS_ENDPOINT is required.");
var busNamespace = Environment.GetEnvironmentVariable("SERVICEBUS_NAMESPACE")
    ?? throw new InvalidOperationException("SERVICEBUS_NAMESPACE is required.");

using var cosmos = new CosmosClient(cosmosEndpoint, new DefaultAzureCredential());
var container = cosmos.GetContainer("saas", "records");

await using var client = new ServiceBusClient(busNamespace, new DefaultAzureCredential());
await using var processor = client.CreateProcessor("work", new ServiceBusProcessorOptions
{
    MaxConcurrentCalls = 8,
    AutoCompleteMessages = false
});

processor.ProcessMessageAsync += async args =>
{
    var evt = args.Message.Body.ToObjectFromJson<OrderAccepted>()
        ?? throw new InvalidOperationException("Invalid event body.");

    var partition = new PartitionKey(evt.TenantId);
    var batch = container.CreateTransactionalBatch(partition);

    batch.CreateItem(new
    {
        id = $"inbox:{evt.EventId}",
        tenantId = evt.TenantId,
        type = "inbox",
        eventId = evt.EventId
    });

    batch.UpsertItem(new
    {
        id = $"order:{evt.OrderId}",
        tenantId = evt.TenantId,
        type = "order",
        status = "processing",
        version = evt.Version
    });

    using var result = await batch.ExecuteAsync(args.CancellationToken);

    if (result.IsSuccessStatusCode)
    {
        await args.CompleteMessageAsync(args.Message, args.CancellationToken);
    }
    else if (result.StatusCode == HttpStatusCode.Conflict)
    {
        await args.CompleteMessageAsync(args.Message, args.CancellationToken);
    }
    else
    {
        throw new RequestFailedException((int)result.StatusCode, "Cosmos batch failed.");
    }
};

processor.ProcessErrorAsync += args =>
{
    Console.Error.WriteLine($"Service Bus error: {args.Exception.Message}");
    return Task.CompletedTask;
};

await processor.StartProcessingAsync();
await Task.Delay(Timeout.Infinite);

public sealed record OrderAccepted(
    string EventId,
    string TenantId,
    string OrderId,
    int Version);

Notice the side effect and inbox marker share a partition and transaction; if the side effect spans other systems, use an additional idempotent workflow rather than claiming this transaction covers them.

The sample uses simple host lifetime and error output to make the flow visible. A production worker should use the .NET Generic Host, health checks, structured logging, dependency injection, cancellation-aware shutdown, and explicit retry and dead-letter policies.

Key flows

The synchronous flow commits the resource and outbox record together. If the commit fails, the API does not acknowledge the command. The outbox dispatcher can retry publishing without losing the committed intent.

sequenceDiagram
    actor Client
    participant Api as Container Apps API
    participant Cosmos as Cosmos DB
    participant Dispatcher as Outbox dispatcher
    participant Bus as Service Bus
    Client->>Api: Submit command
    Api->>Api: Authenticate and authorize tenant
    Api->>Cosmos: Transactional batch resource and outbox
    Cosmos-->>Api: Commit
    Api-->>Client: 202 Accepted with operation URL
    Dispatcher->>Cosmos: Read pending outbox events
    Dispatcher->>Bus: Send event with stable event ID
    Bus-->>Dispatcher: Acknowledge send
    Dispatcher->>Cosmos: Mark dispatched or advance checkpoint

Caption: The outbox closes the database-to-queue dual-write gap by persisting intent before dispatch.

The worker flow assumes at-least-once delivery. It acknowledges only after its durable work succeeds or after it recognizes a duplicate. A permanent validation or business-rule failure should be dead-lettered with a reason, not retried indefinitely.

sequenceDiagram
    participant Bus as Service Bus
    participant Worker as Container Apps worker
    participant Cosmos as Cosmos DB
    Bus->>Worker: Deliver message
    Worker->>Worker: Validate envelope
    Worker->>Cosmos: Transactional batch inbox and projection
    alt New event
        Cosmos-->>Worker: Commit
        Worker->>Bus: Complete message
    else Duplicate event
        Cosmos-->>Worker: Inbox conflict
        Worker->>Bus: Complete message
    else Transient failure
        Cosmos-->>Worker: Failure
        Worker->>Bus: Abandon or retry
    end

Caption: Acknowledge after durable processing, and route repeatedly failing messages to the dead-letter queue.

Build and release

Bicep should create the environment and application dependencies, not bake environment-specific secrets into container images. The following is a compact starting point for one region. It uses managed identities, an Azure Container Registry pull identity, a Premium Service Bus namespace, and a Cosmos DB container partitioned by tenant. The image name and region are parameters; production deployments should also define networking, diagnostic settings, resource locks, and policy-compliant naming.

param location string = resourceGroup().location
param image string
param acrName string
param environmentName string = 'saas-prod'
param cosmosName string
param serviceBusName string

var acrPullRole = '7f951dda-4ed3-4680-a7ca-43fe172d538d'
var cosmosDataContributorRole = '00000000-0000-0000-0000-000000000002'
var serviceBusSenderRole = '69a216fc-b8fb-44d8-bc22-1f3c2cd27a39'
var serviceBusReceiverRole = '4f6d3b9b-027b-4f4c-9141-8e5ca3e9c5a6'

resource acr 'Microsoft.ContainerRegistry/registries@2023-07-01' existing = {
  name: acrName
}

resource pullIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: '${environmentName}-pull'
  location: location
}

resource acrPull 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(acr.id, pullIdentity.id, acrPullRole)
  scope: acr
  properties: {
    principalId: pullIdentity.properties.principalId
    principalType: 'ServicePrincipal'
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', acrPullRole)
  }
}

resource logs 'Microsoft.OperationalInsights/workspaces@2022-10-01' = {
  name: '${environmentName}-logs'
  location: location
  properties: {
    sku: { name: 'PerGB2018' }
    retentionInDays: 30
  }
}

resource env 'Microsoft.App/managedEnvironments@2024-03-01' = {
  name: environmentName
  location: location
  properties: {
    appLogsConfiguration: {
      destination: 'log-analytics'
      logAnalyticsConfiguration: {
        customerId: logs.properties.customerId
        sharedKey: logs.listKeys().primarySharedKey
      }
    }
  }
}

resource cosmos 'Microsoft.DocumentDB/databaseAccounts@2024-05-15' = {
  name: cosmosName
  location: location
  kind: 'GlobalDocumentDB'
  properties: {
    databaseAccountOfferType: 'Standard'
    consistencyPolicy: {
      defaultConsistencyLevel: 'Session'
    }
    locations: [
      {
        locationName: location
        failoverPriority: 0
        isZoneRedundant: true
      }
    ]
  }
}

resource db 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases@2024-05-15' = {
  parent: cosmos
  name: 'saas'
  properties: {
    resource: { id: 'saas' }
  }
}

resource records 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers@2024-05-15' = {
  parent: db
  name: 'records'
  properties: {
    resource: {
      id: 'records'
      partitionKey: {
        paths: ['/tenantId']
        kind: 'Hash'
      }
      indexingPolicy: {
        indexingMode: 'consistent'
        automatic: true
        includedPaths: [{ path: '/*' }]
        excludedPaths: [{ path: '/"_etag"/?' }]
      }
    }
    options: {
      autoscaleSettings: {
        maxThroughput: 4000
      }
    }
  }
}

resource bus 'Microsoft.ServiceBus/namespaces@2022-10-01-preview' = {
  name: serviceBusName
  location: location
  sku: {
    name: 'Premium'
    tier: 'Premium'
    capacity: 1
  }
}

resource queue 'Microsoft.ServiceBus/namespaces/queues@2022-10-01-preview' = {
  parent: bus
  name: 'work'
  properties: {
    requiresDuplicateDetection: true
    duplicateDetectionHistoryTimeWindow: 'PT10M'
    deadLetteringOnMessageExpiration: true
    maxDeliveryCount: 10
  }
}

output cosmosEndpoint string = cosmos.properties.documentEndpoint
output serviceBusNamespace string = bus.properties.serviceBusEndpoint

Notice that the container autoscale maximum, Service Bus messaging unit, retention, and API versions must be validated against the target region, subscription quotas, and workload before rollout; they are starting assumptions, not capacity guarantees.

The two Service Bus role definition IDs in this compact example should be verified against the role names in the target Azure environment before deployment. In production, assign Azure Service Bus Data Sender only to the API or dispatcher that publishes, and Azure Service Bus Data Receiver only to the worker. Assign Cosmos DB Built-in Data Contributor at the narrowest practical scope. A production Bicep module should create those role assignments explicitly and ensure applications tolerate role-assignment propagation delay.

Use a GitHub Actions workflow with workload identity federation for Azure login. Do not store a long-lived Azure client secret in repository secrets.

name: build-and-deploy

on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read
  id-token: write

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-dotnet@v4
        with:
          dotnet-version: '10.0.x'
      - run: dotnet restore
      - run: dotnet test --configuration Release --no-restore

  deploy-staging:
    if: github.event_name == 'push'
    needs: test
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - uses: actions/checkout@v4
      - uses: azure/login@v2
        with:
          client-id: ${{ vars.AZURE_CLIENT_ID }}
          tenant-id: ${{ vars.AZURE_TENANT_ID }}
          subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}
      - name: Build and push image
        run: |
          az acr build \
            --registry "$ACR_NAME" \
            --image "saas-api:${GITHUB_SHA}" \
            --file src/Api/Dockerfile .
        env:
          ACR_NAME: ${{ vars.ACR_NAME }}
      - name: Validate and deploy infrastructure
        run: |
          az deployment group what-if \
            --resource-group "$RESOURCE_GROUP" \
            --template-file infra/main.bicep \
            --parameters image="$IMAGE" acrName="$ACR_NAME" cosmosName="$COSMOS_NAME" serviceBusName="$BUS_NAME"
          az deployment group create \
            --resource-group "$RESOURCE_GROUP" \
            --template-file infra/main.bicep \
            --parameters image="$IMAGE" acrName="$ACR_NAME" cosmosName="$COSMOS_NAME" serviceBusName="$BUS_NAME"
        env:
          RESOURCE_GROUP: ${{ vars.STAGING_RESOURCE_GROUP }}
          IMAGE: ${{ vars.STAGING_IMAGE }}
          ACR_NAME: ${{ vars.ACR_NAME }}
          COSMOS_NAME: ${{ vars.STAGING_COSMOS_NAME }}
          BUS_NAME: ${{ vars.STAGING_BUS_NAME }}
      - name: Update API image
        run: az containerapp update --name saas-api --resource-group "$RESOURCE_GROUP" --image "$IMAGE"
        env:
          RESOURCE_GROUP: ${{ vars.STAGING_RESOURCE_GROUP }}
          IMAGE: ${{ vars.STAGING_IMAGE }}

Notice that the workflow deploys only from main; production should use a separately protected environment, approval gates, and the same immutable commit image.

The workflow is illustrative: define the image parameter as a full ACR login-server path in a real deployment, and split infrastructure creation from application rollout when their permissions or lifecycles differ. Prefer a staged Container Apps revision rollout with health checks and a rollback procedure over an unobserved image replacement.

The test strategy should be layered:

  • Unit: domain rules, tenant authorization policies, event serialization, and retry classification.
  • Integration: Cosmos transactional batches, partition-key behavior, Service Bus sending/receiving, and managed-identity authorization in a non-production Azure environment.
  • Contract: versioned event schemas and compatibility checks between dispatcher and worker.
  • Load: representative tenant distribution, burst traffic, queue backlog recovery, and Cosmos RU consumption. Include a deliberately large tenant rather than testing only uniformly small tenants.

Observability

Instrument the API, dispatcher, and worker. Carry a trace context through the outbox event into Service Bus application properties so the worker can continue the distributed trace. Add tenantId only in a privacy-reviewed, appropriately bounded form; never use tenant IDs, user IDs, or message IDs as metric labels because high-cardinality dimensions make metrics expensive and hard to query.

A structured log event should capture enough context to investigate a failure without logging credentials, message bodies, or personal data:

{
  "timestamp": "2026-11-04T12:00:00Z",
  "level": "Information",
  "service": "orders-api",
  "operation": "CreateOrder",
  "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
  "tenantId": "ten_42",
  "orderId": "ord_01J9A7",
  "outcome": "accepted",
  "durationMs": 82
}

Use .NET OpenTelemetry instrumentation and export using OTLP to the configured collector or Azure Monitor OpenTelemetry distribution. Add the relevant OpenTelemetry packages to the application.

using System.Diagnostics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource.AddService("orders-api"))
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSource("SaaS.Orders")
        .AddOtlpExporter());

var app = builder.Build();

var source = new ActivitySource("SaaS.Orders");
app.MapPost("/orders", () =>
{
    using var activity = source.StartActivity("orders.accept");
    activity?.SetTag("app.operation", "create-order");
    return Results.Accepted("/operations/op-123");
});

app.Run();

Notice that the custom activity joins automatic ASP.NET Core and HTTP client instrumentation; configure the OTLP endpoint and authentication for the actual deployment.

Track RED (rate, errors, duration) for HTTP endpoints and message processing. Track USE (utilization, saturation, errors) for the resources that can constrain the service:

SignalUseful measurements
API REDRequests/s by route template, 5xx and timeout rate, p50/p95/p99 latency
Worker REDMessages completed/s, processing failures, processing duration
Service BusActive message count, age of oldest message, dead-letter count, throttling
Cosmos DBRU consumption and throttling responses, request latency, availability
Container Apps USECPU and memory utilization, replica count, scale limits, restarts
Platform USELog ingestion volume, storage and network saturation where instrumented
SLIInitial SLOMeasurement
API availability99.9% monthlySuccessful eligible requests divided by eligible requests, excluding agreed client errors
API latency95% under 500 msServer-side latency for successful API requests, by route class
Accepted-command durability99.99% of acknowledged writes remain representedReconcile accepted command records and outbox records; validate with fault tests
Queue completion99% of valid messages complete within 5 minutesMessage enqueue time to successful consumer completion
RecoveryRestore service within 4 hours for a regional incidentTested recovery exercise, not an inferred platform capability

Page on sustained user-visible API error-budget burn, API availability failure, a rapidly growing oldest-message age that threatens the completion SLO, Cosmos throttling that persists despite backoff, and dead-letter growth above an agreed threshold. Alert on symptoms and customer impact, not every replica or transient retry. Dashboards should separate API health, async backlog, tenant-aware support diagnostics, and deployment/revision health.

Security and compliance

Use separate identities and permissions for the API, dispatcher, and worker. The API should not have receive rights on the queue; the worker should not have send rights unless its role requires them. Use managed identities for Cosmos DB and Service Bus access. ACR pull uses a dedicated identity or a clearly scoped app identity. Avoid connection strings and account keys in application settings.

The public API needs TLS, authentication, tenant authorization, request-size limits, rate limiting, and input validation. Keep worker ingress internal or disabled. For private network access to Cosmos DB and Service Bus, add a VNet-integrated Container Apps environment, private endpoints, private DNS, and an explicit outbound strategy. That is a meaningful infrastructure and operational step, not a checkbox: test name resolution, deployment access, image pulls, and observability paths.

Encrypt data in transit and at rest using the service capabilities and key-management policy appropriate to the product. Use Key Vault only for secrets that cannot be eliminated with identity-based access. Define retention and deletion for tenant data, outbox records, dead-letter messages, logs, and backups. A dead-letter queue can contain sensitive payloads and needs the same access and retention discipline as the primary store.

For compliance, map controls to the actual product obligations and customer contracts. Do not imply that selecting Azure services alone makes an application compliant. Maintain a data inventory, access reviews, incident response procedures, audit trails for sensitive actions, and tested deletion and restore workflows.

Cost

This is an assumption-based monthly estimate, not an Azure price quote. Actual cost depends on region, negotiated pricing, uptime, throughput, selected networking, log volume, retention, and support plan. Recheck the Azure Pricing Calculator and measured usage before committing to a budget.

AreaPlanning allowance (USD/month)Assumption
Container Apps$250–$700API and worker scale with demand; modest baseline replicas and burst capacity
Cosmos DB$300–$800Single-region, zone-redundant configuration where available, autoscale throughput, 200 GB data assumption
Service Bus$700–$1,100One Premium messaging unit, queue traffic, and availability-oriented tier
Container Registry$30–$100Private image storage and routine image pulls
Azure Monitor and logs$100–$40030-day retention, moderate structured logs, sampled traces
Networking and Key Vault$100–$400Allowance for private connectivity and supporting resources, if enabled
Estimated total$1,480–$3,500Excludes taxes, support plan, DR region, and exceptional egress or log volume

The budget is sensitive to two design choices: Premium Service Bus for the intended availability characteristics and the amount of Cosmos DB throughput actually consumed. A low-traffic system may still pay for a minimum premium tier; a high-churn log strategy can outgrow the application compute bill. Put budget alerts on subscriptions and resource groups, and review cost per tenant and per successfully completed unit of work.

Failure modes, resilience and disaster recovery

FailureExpected behaviorOperational response
API replica crashesIn-flight request may fail; client retries with an idempotency keyMonitor restarts and 5xx; make command submission safe to retry
Cosmos throttlesSDK retries within configured bounds; API may return a controlled retryable responseInspect RU use and hot partitions; change queries or throughput based on evidence
Service Bus unavailableOutbox retains unsent intent; API can continue only while database capacity and product policy allowAlert on oldest pending outbox event and resume dispatch after recovery
Worker crashes after database commitMessage redelivers; inbox marker prevents duplicate side effectsVerify duplicate handling and delivery count
Poison messageRepeated delivery eventually dead-lettersInspect reason and schema; fix or explicitly replay with auditability
One tenant is unusually hotPartition-level contention may affect that tenant and its operationsIdentify tenant skew; throttle, batch, or redesign partition strategy
Regional outageSingle-region baseline may be unavailableFollow documented restore/failover plan; do not claim multi-region recovery before testing

Service Bus Premium availability-zone support and Cosmos DB zone redundancy should be confirmed for the selected region and configuration. Automatic service resiliency does not replace application retry limits, poison-message handling, or restore drills.

For regional disaster recovery, first decide the required recovery point objective (RPO) and recovery time objective (RTO). A backup-and-restore strategy is simpler but usually has a longer recovery time. A second Cosmos DB region and a regional messaging strategy can reduce recovery time, but introduce routing, data consistency, failover, and duplicate-processing concerns. Keep recovery steps versioned, test them on a schedule, and verify that restored identities, networking, DNS, images, and configuration work—not merely that data can be copied.

Evolving to the next scale

Do not split the modular monolith merely because the architecture diagram has several boxes. Split a capability when an independent deployment boundary solves a real constraint: different scaling behavior, ownership, security boundary, release cadence, or failure isolation. At this scale, the API and worker already have separate runtime scaling while sharing a repository and domain contracts.

SignalChange to consider
Sustained queue age breaches the completion SLOIncrease worker concurrency or replicas after validating Cosmos and downstream capacity; then assess workload-specific queues
A tenant dominates partition trafficIntroduce a tenant-specific partitioning strategy or workload isolation; this may require data migration
Read traffic drives RU costAdd carefully invalidated caches for stable, non-sensitive reads; consider dedicated read models for proven hot queries
Event consumers need independent release cyclesSplit a worker or bounded context into a separate Container Apps app with its own contract and operational ownership
Regional recovery objectives tightenDesign and rehearse a multi-region topology with explicit data and messaging failover
Log volume obscures useful signalsApply sampling, structured retention policies, and per-category ingestion controls before increasing retention

Caching is not a default correctness improvement. Define what can be stale, how invalidation works, and whether tenant scope is part of the cache key. Likewise, a second queue is not isolation if every worker still saturates the same Cosmos partition or downstream API.

Phased implementation plan

  1. Agree on product and operational contracts. Define tenant authorization, request idempotency, async status semantics, message schema ownership, data retention, SLIs, and recovery objectives. Review access patterns before fixing the Cosmos partition key.
  2. Build a thin vertical slice. Deploy one API, one worker, one Service Bus queue, and one Cosmos container to a development environment. Use managed identities and verify the full publish, process, retry, and duplicate-delivery path.
  3. Establish infrastructure and delivery. Create reusable Bicep modules, separate staging and production configuration, image scanning, protected deployment environments, and immutable image tagging. Add integration and contract tests before adding more capabilities.
  4. Prove operational behavior. Create RED/USE dashboards, queue-age and dead-letter alerts, structured logs, traces, budget alerts, and a runbook. Load test tenant skew, throttling, message bursts, and worker recovery.
  5. Harden and launch. Add network restrictions and private endpoints if required, validate regional availability features, perform a restore exercise, and conduct a production readiness review with the on-call owner.
  6. Scale from evidence. Review SLO performance, RU consumption, queue age, tenant distribution, and cost per workload. Change partitioning, caching, service boundaries, or regional topology only when an observed constraint or product requirement justifies the added complexity.

The central trade-off is deliberate: Cosmos DB and Service Bus make asynchronous, tenant-scoped workloads manageable, but they do not remove distributed-systems failure modes. Durable outbox intent, idempotent consumers, explicit tenant boundaries, and observable recovery paths are the architecture—not optional refinements.


This article was generated with AI. #AIGenerated