1. Context and scale
Consider a B2B project-management SaaS: organizations manage projects, invite colleagues, and request exports. Traffic is interactive, tenant sizes vary, and losing an acknowledged write is unacceptable.
The three tiers are:
- Presentation: a browser application delivered through Amazon CloudFront and Amazon S3.
- Application: a modular Python monolith on Amazon ECS with AWS Fargate, plus background workers built from the same repository.
- Data: Amazon Aurora PostgreSQL for durable state and Amazon ElastiCache for Valkey for disposable cached data.
Amazon SQS carries background work. It is not a fourth business tier; it decouples application execution.
Use supported, stable baselines: Python 3.13, Aurora PostgreSQL-compatible 17, and ElastiCache for Valkey 8.0. Confirm engine availability in the deployment Region, select approved minor versions, and pin application dependencies and container digests. Do not treat “latest” as a deployment strategy.
Planning assumptions
Every capacity and budget number below is an assumption, not a measured benchmark.
| Dimension | Assumption |
|---|---|
| Users | 40,000 daily active users across 800 tenants |
| Requests | 120 requests/s during busy periods; 600 requests/s short bursts |
| Workload | 85% reads, 15% writes; exports run asynchronously |
| Data volume | 300 GB initially; approximately 15 GB monthly growth |
| Team | Eight engineers across two teams, with shared platform ownership |
| Availability | One AWS Region, application capacity across two Availability Zones |
| Budget | Approximately US$2,500/month initially, excluding engineering and support plans |
Start with four API tasks and two worker tasks. Those counts are hypotheses to validate with representative traffic, not guarantees that a particular Fargate size handles a particular throughput.
The main architectural decision is to keep tenant correctness in PostgreSQL and tenant acceleration in ElastiCache. A cache failure may slow the product; it must not change who can read a project.
2. Requirements
Functional requirements
| Requirement | Architectural consequence |
|---|---|
| Organizations own projects and memberships | Tenant identifiers appear in keys, constraints, and authorization |
| Users access only authorized organizations | Verify identity and resolve membership before opening tenant-scoped work |
| Project creation supports safe retries | Persist an idempotency key with the created project |
| Exports run outside HTTP request lifetimes | Publish durable jobs through an outbox and SQS |
| Users see current data after their own writes | Return authoritative write results; bypass cache where freshness is required |
| Operators investigate tenant-specific failures | Correlate logs and traces without using tenant IDs as metric dimensions |
Non-functional requirements
| Requirement | Initial target or constraint |
|---|---|
| API availability | 99.9% successful eligible requests per calendar month |
| Interactive latency | 95% of eligible API requests complete within 400 ms |
| Data durability | Acknowledged writes survive an instance or Availability Zone failure |
| Tenant isolation | Enforced in application authorization and PostgreSQL row-level security |
| Deployment | Automated staging validation and reversible application releases |
| Regional recovery | Assumed RPO of 15 minutes and RTO of four hours, subject to recovery testing |
| Operations | An on-call engineer can identify the failing tier without database shell access |
Availability excludes client-caused authorization and validation failures, but includes service throttling and dependency failures. Define these classifications in instrumentation rather than editing them during incidents.
Latency applies to synchronous APIs, not export completion. Exports need a separate queue-age and completion-time objective.
Do not promise the regional recovery targets until backup-copy lag and restoration drills demonstrate them.
3. High-level design
flowchart LR
Browser["Browser"]
Cognito["Amazon Cognito"]
Edge["Amazon CloudFront and AWS WAF"]
S3["Amazon S3 web assets"]
ALB["Application Load Balancer"]
API["Amazon ECS Fargate API"]
DB["Aurora PostgreSQL writer and reader"]
Cache["ElastiCache for Valkey"]
Relay["ECS Fargate outbox relay"]
Queue["Amazon SQS and DLQ"]
Worker["ECS Fargate workers"]
Files["Amazon S3 exports"]
Browser --> Edge
Browser --> Cognito
Edge --> S3
Edge --> ALB
ALB --> API
API --> DB
API --> Cache
Relay --> DB
Relay --> Queue
Queue --> Worker
Worker --> DB
Worker --> FilesThe application owns business behavior; Aurora owns truth, while ElastiCache and SQS absorb repeat reads and deferred work.
CloudFront serves immutable frontend assets and routes /api/* to the ALB. Disable caching for authenticated API behaviors unless an endpoint has an explicitly reviewed cache policy. Never let an edge-cache key accidentally collapse responses across users.
Place the ALB in public subnets and Fargate tasks in private subnets across two Availability Zones. Aurora and ElastiCache are private. Use an Aurora writer and a provisioned reader in different Availability Zones; the reader initially provides failover capacity, not a blanket destination for application reads.
Keep the application modular: identity and tenancy, projects, exports, and notifications. Each module owns its writes. A shared deployment does not justify arbitrary cross-module table updates.
Workers and the outbox relay use the same image with different commands and IAM task roles. They scale independently without creating independent business services.
Set an application database-connection budget before enabling autoscaling. For example, an assumed ceiling of 12 API tasks with eight pooled connections each consumes 96 connections before workers, migrations, monitoring, and operational headroom.
4. Cloud service mapping
These equivalents describe responsibility, not interchangeable behavior.
| Component | Chosen configuration | AWS service | Azure equivalent | GCP equivalent |
|---|---|---|---|---|
| Static frontend | Private bucket behind CDN | S3 and CloudFront | Blob Storage and Azure Front Door | Cloud Storage and Cloud CDN |
| Edge protection | Managed rules plus application-specific limits | AWS WAF | Azure Front Door WAF | Cloud Armor |
| Application runtime | Private container tasks | ECS Fargate | Azure Container Apps | Cloud Run |
| HTTP ingress | Regional application routing | Application Load Balancer | Application Gateway | Application Load Balancer |
| Relational storage | PostgreSQL-compatible, multi-AZ | Aurora PostgreSQL | Azure Database for PostgreSQL Flexible Server | Cloud SQL for PostgreSQL |
| Distributed cache | Valkey replication group | ElastiCache | Azure Managed Redis | Memorystore |
| Work queue | Standard queue with DLQ | SQS | Service Bus | Pub/Sub |
| Customer identity | Hosted identity provider | Cognito user pools | Microsoft Entra External ID | Identity Platform |
| Secrets | Task-scoped secret retrieval | Secrets Manager | Key Vault | Secret Manager |
| Telemetry | Managed metrics, logs, traces | CloudWatch and AWS X-Ray | Azure Monitor | Google Cloud Observability |
The alternatives differ in networking, availability models, scaling semantics, and pricing. In particular, Aurora is not simply a differently branded managed PostgreSQL instance.
Use AWS-native integration where it removes operational work, but keep business logic dependent on PostgreSQL, HTTP, and explicit queue contracts rather than infrastructure SDKs everywhere.
5. Data design
Use a pooled tenancy model: shared tables with a mandatory tenant_id. PostgreSQL row-level security provides defense in depth; it does not replace membership authorization.
erDiagram
TENANT ||--o{ PROJECT : owns
TENANT ||--o{ OUTBOX_EVENT : owns
PROJECT ||--o{ OUTBOX_EVENT : produces
TENANT {
uuid id PK
text name
}
PROJECT {
uuid tenant_id PK
uuid id PK
text title
text request_key
timestamptz created_at
}
OUTBOX_EVENT {
uuid tenant_id PK
uuid id PK
uuid project_id FK
jsonb payload
timestamptz published_at
}Composite keys prevent an event from referencing a project in another tenant.
The following is a runnable local schema. Production migrations use a dedicated schema-owner identity; application tasks must not own tables or hold BYPASSRLS.
CREATE ROLE saas_app NOLOGIN;
CREATE TABLE tenant (
id uuid PRIMARY KEY,
name text NOT NULL
);
CREATE TABLE project (
tenant_id uuid NOT NULL REFERENCES tenant(id),
id uuid NOT NULL,
title text NOT NULL CHECK (length(title) BETWEEN 1 AND 200),
request_key text NOT NULL CHECK (length(request_key) BETWEEN 1 AND 128),
created_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (tenant_id, id),
UNIQUE (tenant_id, request_key)
);
CREATE TABLE outbox_event (
tenant_id uuid NOT NULL REFERENCES tenant(id),
id uuid NOT NULL,
project_id uuid NOT NULL,
event_type text NOT NULL,
payload jsonb NOT NULL,
occurred_at timestamptz NOT NULL DEFAULT now(),
published_at timestamptz,
PRIMARY KEY (tenant_id, id),
FOREIGN KEY (tenant_id, project_id)
REFERENCES project(tenant_id, id)
);
CREATE INDEX project_listing
ON project (tenant_id, created_at DESC, id);
CREATE INDEX outbox_pending
ON outbox_event (occurred_at)
WHERE published_at IS NULL;
ALTER TABLE project ENABLE ROW LEVEL SECURITY;
ALTER TABLE project FORCE ROW LEVEL SECURITY;
ALTER TABLE outbox_event ENABLE ROW LEVEL SECURITY;
ALTER TABLE outbox_event FORCE ROW LEVEL SECURITY;
CREATE POLICY project_tenant ON project
FOR ALL TO saas_app
USING (
tenant_id = current_setting('app.tenant_id', true)::uuid
)
WITH CHECK (
tenant_id = current_setting('app.tenant_id', true)::uuid
);
CREATE POLICY outbox_tenant ON outbox_event
FOR ALL TO saas_app
USING (
tenant_id = current_setting('app.tenant_id', true)::uuid
)
WITH CHECK (
tenant_id = current_setting('app.tenant_id', true)::uuid
);
GRANT USAGE ON SCHEMA public TO saas_app;
GRANT SELECT, INSERT ON project, outbox_event TO saas_app;
Tenant context must be transaction-local because pooled connections are reused. Missing context fails closed. A compromised application credential can still select a different tenant context, so RLS is not protection against arbitrary application code execution.
The relay needs cross-tenant access to the outbox. Give it an explicit, separately reviewed policy and only the permissions required to claim and mark events—not unrestricted access to business tables.
Retain idempotency records for the documented retry window. This simplified design retains the key with the project; deletion workflows must preserve equivalent deduplication history.
6. Low-level design
classDiagram
class ProjectEndpoint {
createProject()
getProject()
}
class TenantAuthorizer {
resolveMembership()
}
class ProjectService {
create()
get()
}
class ProjectRepository {
insertIdempotently()
find()
}
class ProjectCache {
get()
set()
}
class OutboxRelay {
publishBatch()
}
ProjectEndpoint --> TenantAuthorizer
ProjectEndpoint --> ProjectService
ProjectService --> ProjectRepository
ProjectService --> ProjectCache
OutboxRelay --> ProjectRepositoryAuthorization establishes tenant context; repositories apply it inside each database transaction.
Minimal runnable application
Install the dependencies and start a small HTTP process:
python3.13 -m venv .venv
source .venv/bin/activate
pip install fastapi uvicorn asyncpg redis
# minimal.py
from fastapi import FastAPI
app = FastAPI()
@app.get("/health/live")
async def live():
return {"status": "ok"}
uvicorn minimal:app --host 0.0.0.0 --port 8080
curl --fail http://localhost:8080/health/live
Notice that liveness does not query Aurora or ElastiCache; a dependency outage should not restart every task.
Readiness is separate. Confirm configuration and successful startup, but avoid making optional cache availability an admission requirement.
Realistic runnable transactional slice
This example creates a project and outbox event atomically, then reads through a tenant-scoped cache. It runs as a CLI so the data path can be exercised without inventing an insecure HTTP authentication shortcut.
Start local dependencies and apply the preceding DDL as schema.sql:
docker run -d --name saas-pg \
-e POSTGRES_PASSWORD=local-admin -p 5432:5432 postgres:17
docker run -d --name saas-cache \
-p 6379:6379 valkey/valkey:8.0
until docker exec saas-pg pg_isready -U postgres; do sleep 1; done
docker exec -i saas-pg psql -v ON_ERROR_STOP=1 -U postgres \
< schema.sql
docker exec -i saas-pg psql -v ON_ERROR_STOP=1 -U postgres <<'SQL'
ALTER ROLE saas_app LOGIN PASSWORD 'local-only';
INSERT INTO tenant (id, name)
VALUES ('11111111-1111-1111-1111-111111111111', 'Example');
SQL
export DATABASE_URL='postgresql://saas_app:local-only@localhost/postgres'
export CACHE_URL='redis://localhost:6379'
# project_slice.py
import asyncio
import json
import logging
import os
import uuid
import asyncpg
from redis.asyncio import Redis
from redis.exceptions import RedisError
TENANT = uuid.UUID("11111111-1111-1111-1111-111111111111")
log = logging.getLogger("projects")
async def create_project(pool, tenant, request_key, title):
async with pool.acquire() as conn:
async with conn.transaction():
await conn.execute(
"SELECT set_config('app.tenant_id', $1, true)",
str(tenant),
)
row = await conn.fetchrow(
"""
INSERT INTO project (tenant_id, id, title, request_key)
VALUES ($1, $2, $3, $4)
ON CONFLICT (tenant_id, request_key) DO NOTHING
RETURNING id, title
""",
tenant, uuid.uuid4(), title, request_key,
)
if row is None:
row = await conn.fetchrow(
"""
SELECT id, title FROM project
WHERE tenant_id = $1 AND request_key = $2
""",
tenant, request_key,
)
if row["title"] != title:
raise ValueError("Idempotency key reused with another title")
return row["id"]
event_id = uuid.uuid4()
await conn.execute(
"""
INSERT INTO outbox_event
(tenant_id, id, project_id, event_type, payload)
VALUES ($1, $2, $3, 'project.created', $4::jsonb)
""",
tenant, event_id, row["id"],
json.dumps({
"schema_version": 1,
"event_id": str(event_id),
"tenant_id": str(tenant),
"project_id": str(row["id"]),
}),
)
return row["id"]
async def get_project(pool, cache, tenant, project_id):
key = f"tenant:{tenant}:project:{project_id}:v1"
try:
cached = await cache.get(key)
if cached is not None:
return json.loads(cached)
except (RedisError, ValueError):
log.warning("cache_read_failed")
async with pool.acquire() as conn:
async with conn.transaction(readonly=True):
await conn.execute(
"SELECT set_config('app.tenant_id', $1, true)",
str(tenant),
)
row = await conn.fetchrow(
"""
SELECT id, title FROM project
WHERE tenant_id = $1 AND id = $2
""",
tenant, project_id,
)
if row is None:
return None
result = {"id": str(row["id"]), "title": row["title"]}
try:
await cache.set(key, json.dumps(result), ex=30)
except RedisError:
log.warning("cache_write_failed")
return result
async def main():
logging.basicConfig(level=logging.INFO)
pool = await asyncpg.create_pool(
os.environ["DATABASE_URL"],
min_size=1,
max_size=4,
command_timeout=3,
)
cache = Redis.from_url(
os.environ["CACHE_URL"],
decode_responses=True,
socket_connect_timeout=0.05,
socket_timeout=0.05,
)
try:
project_id = await create_project(
pool, TENANT, "demo-create-001", "Customer rollout"
)
print(await get_project(pool, cache, TENANT, project_id))
finally:
await cache.aclose()
await pool.close()
if __name__ == "__main__":
asyncio.run(main())
python project_slice.py
python project_slice.py
Notice that both runs return the same project, while only the first transaction creates an outbox event.
The CLI tenant constant is a test fixture. In HTTP handlers, verify Cognito token signature, issuer, expiry, token use, and the appropriate audience or client identifier, then resolve active organization membership server-side.
For general payloads, persist a canonical request hash rather than comparing one field. Apply request-size limits before hashing.
| API operation | Success | Important failure |
|---|---|---|
POST /projects with Idempotency-Key | 201; replay returns the original resource | 409 for key reuse with different content |
GET /projects/{id} | 200 | 404 for absent or inaccessible project |
POST /exports | 202 with job identifier | 429 for tenant quota exhaustion |
GET /exports/{id} | 200 with status | Download authorization rechecked before signing |
The illustrative cache permits up to 30 seconds of staleness. Mutation responses come from Aurora. Mutable project details need an explicit freshness policy; simple delete-on-write does not eliminate stale cache-refill races.
7. Key flows
Project creation and durable publication
sequenceDiagram
participant Client
participant API
participant DB as Aurora PostgreSQL
participant Relay as Outbox relay
participant Queue as Amazon SQS
Client->>API: Create project with idempotency key
API->>API: Verify token and tenant membership
API->>DB: Begin transaction and set tenant context
API->>DB: Insert project and outbox event
DB-->>API: Commit
API-->>Client: Created project
Relay->>DB: Claim pending outbox batch
Relay->>Queue: Publish event with stable event ID
Queue-->>Relay: Accepted
Relay->>DB: Mark event publishedThe database commit is the durability boundary; queue publication can happen later.
Use leased outbox claims or small SKIP LOCKED batches. Avoid holding a database transaction open across a slow network call.
The relay can crash after SQS accepts an event but before marking it published. Therefore publication is at least once. Consumers must deduplicate by event ID.
For exports, use a stable S3 object key derived from the job ID and a durable job-state transition. Do not claim exactly-once delivery; design repeated processing to converge on the same result.
Tenant-scoped cached read
sequenceDiagram
participant Client
participant API
participant Cache as ElastiCache
participant DB as Aurora writer
Client->>API: Read project
API->>API: Authorize tenant membership
API->>Cache: Get tenant-scoped key
alt Cache hit
Cache-->>API: Cached project
else Miss or bounded cache failure
API->>DB: Read within tenant-scoped transaction
DB-->>API: Project
API->>Cache: Best-effort set with TTL
end
API-->>Client: Project representationCache access happens only after authorization, and cache failure falls back within a bounded timeout.
Use TTL jitter and request coalescing for hot keys. During a cache outage, enforce concurrency limits so bypass traffic cannot overwhelm Aurora.
A cached object is not evidence of membership. Revoking membership must take effect independently of object-cache expiry.
8. Build and release
Provision separate production and staging environments, preferably in separate AWS accounts. Keep Terraform state encrypted and access-controlled, and review plans before applying production changes.
This Terraform excerpt assumes networking, IAM roles, secrets, ALB listeners, and Aurora already exist in surrounding modules. It shows the deployment and cache decisions rather than pretending to be a complete environment.
terraform {
required_version = ">= 1.10, < 2.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 6.0"
}
}
}
resource "aws_elasticache_replication_group" "cache" {
replication_group_id = "saas-cache"
description = "Disposable tenant-scoped cache"
engine = "valkey"
engine_version = "8.0"
node_type = "cache.t4g.small"
num_cache_clusters = 2
automatic_failover_enabled = true
multi_az_enabled = true
at_rest_encryption_enabled = true
transit_encryption_enabled = true
subnet_group_name = var.cache_subnet_group
security_group_ids = [var.cache_security_group_id]
user_group_ids = [var.cache_user_group_id]
}
resource "aws_ecs_task_definition" "api" {
family = "saas-api"
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = "1024"
memory = "2048"
execution_role_arn = var.execution_role_arn
task_role_arn = var.api_task_role_arn
container_definitions = jsonencode([{
name = "api"
image = var.image_digest_uri
essential = true
command = ["uvicorn", "service:app", "--host", "0.0.0.0", "--port", "8080"]
portMappings = [{
containerPort = 8080
protocol = "tcp"
}]
secrets = [
{ name = "DATABASE_URL", valueFrom = var.database_secret_arn },
{ name = "CACHE_URL", valueFrom = var.cache_secret_arn }
]
logConfiguration = {
logDriver = "awslogs"
options = {
awslogs-group = var.log_group
awslogs-region = var.region
awslogs-stream-prefix = "api"
}
}
}])
}
resource "aws_ecs_service" "api" {
name = "saas-api"
cluster = var.cluster_arn
task_definition = aws_ecs_task_definition.api.arn
desired_count = 4
launch_type = "FARGATE"
deployment_minimum_healthy_percent = 100
deployment_maximum_percent = 200
health_check_grace_period_seconds = 60
deployment_circuit_breaker {
enable = true
rollback = true
}
network_configuration {
subnets = var.private_subnet_ids
security_groups = [var.api_security_group_id]
assign_public_ip = false
}
load_balancer {
target_group_arn = var.api_target_group_arn
container_name = "api"
container_port = 8080
}
lifecycle {
ignore_changes = [desired_count, task_definition]
}
}
Notice that release automation owns task-definition revisions, while Terraform owns infrastructure; the ignored fields make that ownership explicit.
Add autoscaling with a tested maximum, not an unlimited response to CPU pressure. Include rolling-deployment overlap in the connection budget.
A GitHub Actions release pipeline can promote one immutable image through staging and production:
name: release
on:
push:
branches: [main]
permissions:
contents: read
id-token: write
concurrency:
group: production-release
cancel-in-progress: false
env:
AWS_REGION: us-east-1
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- run: pip install -r requirements-dev.lock
- run: pytest -q
publish:
needs: test
runs-on: ubuntu-latest
outputs:
image: ${{ steps.image.outputs.uri }}
steps:
- uses: actions/checkout@v4
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ vars.BUILD_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}
- uses: aws-actions/amazon-ecr-login@v2
- id: image
env:
REPOSITORY_URI: ${{ vars.ECR_REPOSITORY_URI }}
run: |
docker build -t "$REPOSITORY_URI:$GITHUB_SHA" .
docker push "$REPOSITORY_URI:$GITHUB_SHA"
DIGEST=$(aws ecr describe-images \
--repository-name "${REPOSITORY_URI##*/}" \
--image-ids imageTag="$GITHUB_SHA" \
--query 'imageDetails[0].imageDigest' --output text)
echo "uri=$REPOSITORY_URI@$DIGEST" >> "$GITHUB_OUTPUT"
deploy:
needs: publish
strategy:
max-parallel: 1
fail-fast: true
matrix:
environment: [staging, production]
environment: ${{ matrix.environment }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ vars.DEPLOY_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}
- uses: aws-actions/amazon-ecs-render-task-definition@v1
id: render
with:
task-definition: deploy/api-task.json
container-name: api
image: ${{ needs.publish.outputs.image }}
- uses: aws-actions/amazon-ecs-deploy-task-definition@v2
with:
task-definition: ${{ steps.render.outputs.task-definition }}
service: ${{ vars.ECS_SERVICE }}
cluster: ${{ vars.ECS_CLUSTER }}
wait-for-service-stability: true
- run: python ci/smoke.py
env:
BASE_URL: ${{ vars.BASE_URL }}
Notice that environment protection supplies the production approval boundary; commit reviewed action SHAs in the actual repository.
The pipeline assumes committed task templates, tests, and smoke checks. Treat production approval as conditional on successful staging verification, not merely elapsed deployment time.
Run schema changes in a separate, approved migration task with a DDL-capable role before compatible application releases. Use expand–migrate–contract: add compatible structures, deploy readers and writers, backfill, then remove obsolete structures in a later release.
| Test layer | What it must prove |
|---|---|
| Unit | Authorization decisions, idempotency validation, job-state transitions |
| Integration | Real PostgreSQL RLS, concurrent retries, outbox atomicity, cache outage behavior |
| Contract | OpenAPI compatibility and versioned queue-event schemas |
| Load | Representative tenant skew, cold cache, rollout overlap, worker backlogs |
| Recovery | Database failover, task loss, queue redelivery, backup restoration |
9. Observability
Instrument the user journey, not just container health.
Log request completion, authorization rejection, durable state transitions, retries, and dependency failures. Include trace and request identifiers, deployment version, route template, outcome, and duration. Tenant identifiers belong only in access-controlled logs when operationally necessary—not high-cardinality metric labels.
{
"timestamp": "2026-11-02T10:15:30Z",
"level": "INFO",
"service": "saas-api",
"environment": "production",
"event": "http_request_completed",
"route": "/projects/{id}",
"method": "GET",
"status_code": 200,
"duration_ms": 84,
"tenant_ref": "tenant_opaque_reference",
"trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
"request_id": "req_7cb12",
"cache_result": "hit",
"deployment": "git_commit"
}
Never log tokens, connection strings, raw request bodies, or signed download URLs.
OpenTelemetry tracing
Export OTLP to an AWS Distro for OpenTelemetry collector sidecar or private collector service. Configure the collector to forward traces to AWS X-Ray and keep exporter failures off the request path.
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-grpc \
opentelemetry-instrumentation-fastapi \
opentelemetry-instrumentation-asyncpg \
opentelemetry-instrumentation-redis
# telemetry.py
import os
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import (
OTLPSpanExporter,
)
from opentelemetry.instrumentation.asyncpg import AsyncPGInstrumentor
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.redis import RedisInstrumentor
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.trace.sampling import (
ParentBased,
TraceIdRatioBased,
)
def configure_telemetry(app):
provider = TracerProvider(
resource=Resource.create({
"service.name": os.getenv("OTEL_SERVICE_NAME", "saas-api"),
"service.version": os.getenv("RELEASE_SHA", "local"),
}),
sampler=ParentBased(TraceIdRatioBased(0.1)),
)
provider.add_span_processor(
BatchSpanProcessor(OTLPSpanExporter())
)
trace.set_tracer_provider(provider)
AsyncPGInstrumentor().instrument()
RedisInstrumentor().instrument()
FastAPIInstrumentor.instrument_app(
app,
excluded_urls="health/live,health/ready",
)
return provider
Notice that tracing is initialized before database and cache clients; the 10% head-sampling rate is an initial assumption, not an error-retention guarantee.
Call this during application construction, set OTEL_EXPORTER_OTLP_ENDPOINT, and shut down the provider during application lifespan cleanup. Use a local insecure OTLP endpoint only for a same-task collector; protect network transport otherwise.
Explicitly propagate trace context through outbox metadata and SQS message attributes. Link worker spans to originating operations when long queue delays make one continuous trace misleading.
Metrics and objectives
| Area | Metrics |
|---|---|
| API RED | Request rate, server-error ratio, duration histogram by route template |
| Worker RED | Completion rate, processing failures, processing duration |
| Fargate USE | CPU and memory utilization, concurrency saturation, task exits |
| Aurora USE | CPU, connection utilization, connection waits, lock waits, storage latency |
| ElastiCache USE | Engine CPU, memory utilization, evictions, connections, replication lag |
| Queue health | Oldest-message age, visible backlog, retries, DLQ growth |
| Outbox health | Oldest unpublished event age and pending count |
| SLI | Initial SLO | Measurement |
|---|---|---|
| API availability | 99.9% monthly | Eligible requests without service failure |
| API latency | 95% below 400 ms | Eligible request duration |
| Export completion | 99% within five minutes | Accepted jobs reaching terminal success |
| Publication delay | 99% within 30 seconds | Commit-to-publish event age |
Maintain an overview dashboard, a dependency dashboard, and a deployment-comparison dashboard.
Page on sustained error-budget burn, exhausted database connections with user impact, no healthy API capacity, and queue age threatening the completion objective. Ticket slow storage growth and cache-hit degradation. Do not page merely because CPU briefly rises.
10. Security and compliance
Authenticate with Cognito, but keep tenant authorization in the application. An authenticated user can belong to several organizations; a token claim alone may not reflect a recent membership revocation.
Apply these boundaries:
- ALB accepts application traffic only from the intended CloudFront origin path; combine origin verification with appropriate network restrictions.
- API tasks accept traffic only from the ALB security group.
- Aurora accepts connections only from approved application, worker, relay, and migration security groups.
- ElastiCache uses TLS and an ACL user restricted to required commands and key patterns.
- Separate IAM task roles grant API, relay, and workers only their required AWS actions.
- The execution role pulls images and retrieves startup secrets; it is not the application’s general-purpose identity.
Use TLS with server identity validation for Aurora and ElastiCache connections. Encryption flags on the service do not prove clients validate certificates.
Secrets injected at task startup do not automatically refresh inside a running process. Coordinate credential rotation with connection renewal and task replacement.
Keep buckets private and use CloudFront origin access control. Generate short-lived export URLs only after checking current authorization.
For compliance, define data classification, residency, retention, deletion, and audit access before claiming any certification. Tenant deletion must cover exports, operational stores, and eventual backup expiry. Record these semantics explicitly rather than promising immediate erasure from immutable backups.
11. Cost
The following is an assumption-based monthly estimate, not an AWS quote. It assumes a US Region, approximately 730 hours, modest internet egress, four small API tasks, two workers, a small relay, two Aurora instances, and a two-node cache.
| Cost area | Assumed monthly amount |
|---|---|
| Fargate API, worker, and relay compute | $300 |
| Aurora instances, storage, and I/O allowance | $800 |
| ElastiCache primary and replica | $200 |
| ALB, CloudFront, and static assets | $160 |
| NAT gateways, endpoints, and network transfer allowance | $180 |
| Logs, metrics, and tracing | $150 |
| Backup copies and export storage | $130 |
| Smaller staging environment | $250 |
| Cognito, WAF, secrets, and miscellaneous services | $180 |
| Estimated total | $2,350 |
The estimate intentionally leaves little room beneath the assumed budget. Authentication usage, egress, Aurora I/O, and verbose logging can materially change the total.
Build a calculator estimate using the selected Region and instance sizes before approval. Compare Aurora Standard and I/O-Optimized using measured I/O rather than assuming one is universally cheaper.
Use cost-allocation tags for environment and workload. Export billing data for trends, but avoid allocating shared infrastructure to tenants with false precision.
Cost controls should preserve correctness: cap log retention, expire exports, sample successful traces, and right-size staging. Do not remove the Aurora failover instance merely to make a spreadsheet fit.
12. Failure modes, resilience and disaster recovery
| Failure | Expected behavior | Recovery mechanism |
|---|---|---|
| API task exits | Some in-flight requests fail | ALB removes target; ECS replaces task; clients retry safe operations |
| Availability Zone fails | Reduced capacity and possible database interruption | Surviving tasks, Aurora failover, cache replica promotion |
| Aurora writer fails | Brief transaction failures | Reconnect through cluster endpoint; bounded whole-transaction retries |
| ElastiCache fails | Read latency and database load rise | Bounded bypass, concurrency limits, cache rebuild |
| SQS delivery repeats | Duplicate processing attempt | Durable consumer deduplication and idempotent effects |
| Worker crashes | Message becomes visible again | Visibility timeout and safe replay |
| Bad deployment | Errors or unhealthy targets rise | Circuit breaker, rollback, deployment halt |
| Region unavailable | Service unavailable until recovery | Restore or promote data and rebuild regional infrastructure |
Aurora storage replication does not remove the need to handle interrupted connections or uncertain commit outcomes. Retry using the original idempotency key.
Set deadlines for every dependency. Retry transient failures with jitter and a small attempt budget; avoid nested retries that multiply load.
For long exports, extend SQS visibility while making progress. Send repeatedly failing jobs to a DLQ and provide a reviewed redrive procedure. A DLQ is not successful completion.
Enable point-in-time recovery and copy recoverable backups to a second Region under a documented schedule. Measure the age of the latest usable recovery point. Backup copying alone may not satisfy the assumed 15-minute RPO; if drills show a gap, fund a warm regional strategy or revise the requirement.
Recovery order is deliberate: restore Aurora, establish secrets and networking, start APIs, verify tenant isolation, start relay and workers, then reopen traffic. Rebuild caches empty.
Test restoration with a new cluster and application smoke tests. A backup job marked successful is not proof that the SaaS can recover.
13. Evolving to the next scale
Scale from evidence, not user-count milestones.
| Observed condition | Next change |
|---|---|
| API CPU saturation with healthy database capacity | Increase task count within the connection budget |
| Database connection churn limits scaling | Evaluate RDS Proxy and its session-state implications |
| Read pressure remains after query tuning | Move explicitly stale-tolerant reads to the Aurora reader |
| Cache stampedes dominate misses | Add coalescing, TTL jitter, and bounded stale serving |
| A few tenants dominate resources | Add tenant quotas and workload isolation |
| Export queues breach objectives | Scale workers by backlog age and isolate heavy job types |
| One module needs independent release ownership | Extract that module behind a stable contract |
| Regional recovery objectives become stricter | Evaluate Aurora Global Database and a rehearsed regional failover plan |
Before adding infrastructure, inspect query plans, index selectivity, transaction duration, and payload size. A missing tenant-leading index is not a reason to shard.
Keep schema ownership boundaries inside the monolith so extraction remains possible. Do not split services while every request still needs a distributed transaction across them.
The next scale is often the same architecture with better limits, measured capacity, and clearer operational ownership.
14. Phased implementation plan
Phase 1: Foundations and one vertical slice
Create accounts, networking, identity integration, and the deployment pipeline. Implement project creation end to end, including RLS and idempotency.
Exit criteria: a staging user can access only authorized tenant data, and concurrent retries create one resource.
Phase 2: Durable asynchronous work
Add the outbox relay, SQS, export workers, and private object storage. Define event schemas and replay procedures.
Exit criteria: killing a worker or relay loses no accepted job and produces no duplicate user-visible effect.
Phase 3: Performance and operational readiness
Add caching only to measured repeat-read paths. Establish connection limits, request deadlines, dashboards, tracing, and error-budget alerts.
Exit criteria: the assumed peak workload survives a cold cache and a rolling deployment without uncontrolled database saturation.
Phase 4: Production resilience
Exercise Aurora failover, restore backups, validate secret rotation, and complete the tenant-deletion workflow. Document ownership for every alert and recovery action.
Exit criteria: the team demonstrates its recovery targets and understands any remaining gap.
Phase 5: Controlled growth
Review cost, noisy tenants, queue age, query performance, and deployment reliability regularly. Increase capacity only with corresponding database and failure-budget checks.
The blueprint succeeds when the team can explain what happens to an acknowledged write, a revoked membership, and a queued job during failure. Fargate, Aurora, and ElastiCache provide the mechanisms; disciplined boundaries make them a dependable SaaS.
This article was generated with AI. #AIGenerated