Skip to content

10 .NET Production Features — Architecture Map

Here's a comprehensive breakdown of all 10 features, from junior through to principal engineer level. I'll build a visual architecture overview first, then walk through each. Now let's go deep on each feature, level by l...

Ekanatha Reddy Urivakili12 min1,805 words
---
title: 10 .NET Production Features — Architecture Map
excerpt: >-
  A comprehensive guide to 10 foundational .NET production features with architecture diagrams, code, and observability best practices for engineers of all levels.
tags:
  - dotnet
  - software-architecture
  - observability
  - resilience
  - api-versioning
seoTitle: 10 Essential .NET Production Features with Architecture Diagrams
seoDescription: Dive deep into 10 key .NET production features with architecture and sequence diagrams, practical code, and observability strategies.
category: tech-stacks
---

Here's a comprehensive breakdown of all 10 features, from junior through to principal engineer level. I'll build a visual architecture overview first, then walk through each.

High-Level Architecture Overview

flowchart TD
    A[Client] -->|HTTP Request| B[API Gateway / Reverse Proxy]
    B -->|Rate Limiting & Routing| C[Service - API Version]
    C -->|Starts Trace & Logs Request| D[Feature Flags Evaluation]
    D --> E[Business Logic]
    E -->|Cache Lookup| F[Cache (Redis)]
    F -->|Cache Miss| G[Database]
    E -->|Calls Downstream APIs| H[Downstream Services]
    H -->|Polly Resilience Policies| H
    E -->|Exceptions Caught| I[Global Exception Handler]
    E -->|SSE Push Updates| J[Browser via SSE]
    B -->|Health Probes| K[Health Check Endpoint]
    C -->|Emit Logs, Metrics, Traces| L[Observability (OpenTelemetry)]

Figure 1: HLD showing integration of the 10 .NET production features.

Interaction Flow: Request to Response Sequence

sequenceDiagram
    participant Client
    participant Gateway
    participant Service
    participant Cache
    participant DB
    participant Downstream
    participant Logging & Tracing
    participant SSEClient

    Client->>Gateway: HTTP Request (v1 or v2)
    Gateway->>Gateway: Apply Rate Limiting
    Gateway->>Service: Forward Request
    Service->>Logging & Tracing: Start Trace & Log Request
    Service->>Feature Flags: Evaluate Flags
    Service->>Cache: Check Cache
    alt Cache Hit
        Cache-->>Service: Return Cached Data
    else Cache Miss
        Service->>DB: Query Database
        DB-->>Service: Return Data
        Service->>Cache: Store Data
    end
    Service->>Downstream: Call APIs with Polly Resilience
    Downstream-->>Service: Return Response
    Service->>Logging & Tracing: Log Response & Metrics
    alt Exception Occurs
        Service->>Service: Global Exception Handler
        Service-->>Client: Problem Details Response
    else
        Service-->>Client: Success Response
    end
    Service->>SSEClient: Push Updates via SSE

Figure 2: Sequence of request handling integrating multiple production features.


① Health Checks + Metrics

What it is: Your app tells the outside world whether it's alive and healthy.

Junior: Add the built-in endpoint so Kubernetes or a load balancer can probe it.

builder.Services.AddHealthChecks()
    .AddSqlServer(connectionString)
    .AddRedis(redisConn);

app.MapHealthChecks("/health");
app.MapHealthChecks("/health/ready"); // is the app ready to serve?
app.MapHealthChecks("/health/live");  // is the process alive?

Notice how the readiness probe checks dependencies whereas the liveness probe only ensures the process is running.

Mid-level: Expose /metrics using prometheus-net or OpenTelemetry so Grafana can scrape CPU, request count, GC pressure, and custom counters you define.

Senior/Principal: Design a tiered health check strategy. /health/live only checks if the process is running (never touches the DB). /health/ready checks DB, Redis, and downstream APIs. This distinction prevents Kubernetes from restarting a pod just because a downstream dependency is slow. You also define degraded states — the app is serving but at reduced capacity — rather than just healthy/unhealthy binary.


② Observability (The Three Pillars)

What it is: The ability to understand system behaviour by examining its outputs — logs, metrics, and distributed traces.

Junior: Know the difference: logs are text events, metrics are numbers over time, traces follow a request across services.

Mid-level: Wire up OpenTelemetry — the vendor-neutral standard.

builder.Services.AddOpenTelemetry()
    .WithTracing(t => t
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddOtlpExporter()) // to Jaeger or Grafana Tempo
    .WithMetrics(m => m
        .AddAspNetCoreInstrumentation()
        .AddPrometheusExporter());

Senior/Principal: You set up correlation — every request gets a TraceId that propagates through HTTP headers (traceparent) so you can reconstruct the full call chain from Gateway → Service A → Service B → DB in one view. You define SLOs (Service Level Objectives): e.g. "p99 latency must stay under 300ms" and alert when breached.


③ Rate Limiting

What it is: Protecting your API from being overwhelmed — either by abuse or traffic spikes.

Junior: Understand that without rate limiting, one bad actor or traffic spike can take your whole service down.

Mid-level: ASP.NET Core 7+ has built-in rate limiting. You can also use YARP (Yet Another Reverse Proxy) at the gateway level.

builder.Services.AddRateLimiter(opts => {
    opts.AddFixedWindowLimiter("fixed", o => {
        o.PermitLimit = 100;
        o.Window = TimeSpan.FromMinutes(1);
    });
    opts.RejectionStatusCode = 429; // Too Many Requests
});

app.UseRateLimiter();
app.MapGet("/api/orders", ...).RequireRateLimiting("fixed");

Senior/Principal: You differentiate policy types — fixed window, sliding window, token bucket, concurrency limiter — and apply them at different levels. Authenticated users get higher limits than anonymous. Premium customers get a separate policy. You also implement distributed rate limiting with Redis so limits work correctly across multiple pod replicas.


Comparison Table: Rate Limiting Algorithms

AlgorithmDescriptionUse CaseComplexityDistributed Support
Fixed WindowLimits requests per fixed time windowSimple, easy to implementLowModerate (can cause bursts)
Sliding WindowUses a rolling window for smoother rate limitsPrecise throttlingMediumModerate
Token BucketTokens represent capacity, refilled at rateFlexible and burst tolerantHighGood
Concurrency LimiterLimits concurrent requestsProtect resource exhaustionMediumComplex

Choose based on your service traffic pattern and requirements for fairness and burst handling.


④ API Versioning

What it is: Evolving your API without breaking existing clients.

Junior: Never change a public API contract without versioning. Adding a field is safe; removing or renaming one is a breaking change.

Mid-level: Use the Asp.Versioning NuGet package.

builder.Services.AddApiVersioning(opts => {
    opts.DefaultApiVersion = new ApiVersion(1, 0);
    opts.AssumeDefaultVersionWhenUnspecified = true;
    opts.ReportApiVersions = true; // returns api-supported-versions header
});

[ApiVersion("1.0")]
[ApiVersion("2.0")]
[Route("api/v{version:apiVersion}/orders")]
public class OrdersController : ControllerBase { ... }

Strategies: URL path (/v1/), query string (?api-version=2), or header (api-version: 2).

Senior/Principal: You maintain a deprecation policy — v1 gets 12 months notice before sunset. You use an API Gateway (YARP) to route traffic so v1 and v2 can be entirely separate services during a migration, gradually shifting traffic. You track which clients still hit v1 via observability before decommissioning.


⑤ Proper Structured Logging

What it is: Machine-parseable logs that you can query, not raw text strings.

Junior: Never do _logger.LogInformation("Order " + orderId + " placed"). Always use structured params.

_logger.LogInformation("Order {OrderId} placed by {UserId}", orderId, userId);

Mid-level: Use Serilog with enrichers and a sink to a log aggregator like Seq or Elasticsearch.

Log.Logger = new LoggerConfiguration()
    .Enrich.WithCorrelationId()      // links log to the request trace
    .Enrich.WithMachineName()
    .WriteTo.Seq("http://seq:5341")
    .CreateLogger();

Senior/Principal: You define log levels as a contract. Information = business events (order placed, user logged in). Warning = degraded behaviour you can recover from. Error = unhandled failures. You never log sensitive data (PII, card numbers). You implement log sampling at high traffic volumes — only logging 10% of successful GET requests but 100% of errors — to reduce cost without losing signal.


⑥ Caching

What it is: Storing expensive computation results in fast memory so you don't hit the database every time.

Junior: Understand the cache invalidation problem — stale data. Know that cache is a copy, not the source of truth.

Mid-level: Use IDistributedCache backed by Redis.

// Cache-aside pattern
var key = $"product:{productId}";
var cached = await cache.GetStringAsync(key);
if (cached != null) return JsonSerializer.Deserialize<Product>(cached);

var product = await db.Products.FindAsync(productId);
await cache.SetStringAsync(key, JsonSerializer.Serialize(product),
    new DistributedCacheEntryOptions {
        AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5)
    });
return product;

Senior/Principal: You choose the right cache pattern: cache-aside (above), write-through (write to cache and DB atomically), or read-through. You think about cache stampede — when the cache expires and 1,000 requests simultaneously hit the DB. You solve it with probabilistic early expiration or a distributed lock. You also consider cache eviction policies (LRU vs TTL) and size limits per service.


⑦ Server-Sent Events (SSE)

What it is: One-way real-time push from server to browser — cheaper than WebSockets when clients only need to receive data (dashboards, notifications, live order status).

Junior: Understand the difference: polling (client asks every X seconds), SSE (server pushes when data changes), WebSockets (bidirectional).

Mid-level: In ASP.NET Core, SSE is just a streaming HTTP response with Content-Type: text/event-stream.

app.MapGet("/events/orders", async (HttpResponse response, CancellationToken ct) => {
    response.Headers["Content-Type"] = "text/event-stream";
    response.Headers["Cache-Control"] = "no-cache";

    while (!ct.IsCancellationRequested) {
        var update = await orderQueue.DequeueAsync(ct);
        await response.WriteAsync($"data: {JsonSerializer.Serialize(update)}\n\n", ct);
        await response.Body.FlushAsync(ct);
    }
});

Senior/Principal: You evaluate SSE vs SignalR vs WebSockets based on scale. SSE is HTTP/1.1 compatible and simpler to scale behind standard load balancers. For fan-out to millions of clients, you introduce a message broker (Redis Pub/Sub, NATS) as the intermediary so any pod can push to any connected client. You also handle reconnection — SSE clients reconnect automatically using the Last-Event-ID header so they don't miss events during network hiccups.


⑧ Feature Management (Feature Flags)

What it is: Deploy code to production without activating it. Toggle features on/off without redeploying.

Junior: Understand that this decouples deployment from release. You ship dark code constantly, then flip a flag to activate it for a subset of users.

Mid-level: Use Microsoft.FeatureManagement.

builder.Services.AddFeatureManagement(); // reads from appsettings.json

// appsettings.json
/*
"FeatureManagement": {
    "NewCheckoutFlow": false,
    "DarkMode": true
}
*/

// Controller
if (await featureManager.IsEnabledAsync("NewCheckoutFlow")) {
    return await newCheckoutService.ProcessAsync(order);
}
return await legacyCheckoutService.ProcessAsync(order);

Senior/Principal: You use targeted feature flags — enable for 5% of users, then 20%, then 100% (canary release). You integrate with a feature flag service (LaunchDarkly, Azure App Config) so flags can be toggled in real time without restarting pods. You use flags for A/B testing, kill switches (instantly disable a broken feature under load), and ring deployments where internal employees see the feature before external customers.


⑨ Exception Handling Strategy

What it is: A centralised, consistent way to catch all unhandled errors and return meaningful, non-leaking responses.

Junior: Never let stack traces reach the client. They expose implementation details and are a security risk.

Mid-level: Use a global middleware in ASP.NET Core 8+ with IExceptionHandler or the Problem Details standard (RFC 7807).

builder.Services.AddProblemDetails();
builder.Services.AddExceptionHandler<GlobalExceptionHandler>();

public class GlobalExceptionHandler : IExceptionHandler {
    public async ValueTask<bool> TryHandleAsync(
        HttpContext ctx, Exception ex, CancellationToken ct) {

        var (status, title) = ex switch {
            ValidationException => (400, "Validation failed"),
            NotFoundException  => (404, "Resource not found"),
            UnauthorizedAccessException => (403, "Forbidden"),
            _ => (500, "An unexpected error occurred")
        };

        ctx.Response.StatusCode = status;
        await ctx.Response.WriteAsJsonAsync(new ProblemDetails {
            Title = title, Status = status,
            Extensions = { ["traceId"] = Activity.Current?.Id }
        }, ct);
        return true;
    }
}

Senior/Principal: You define an exception taxonomy across all services and ensure they all return the same Problem Details shape so client teams have a contract. You combine exception handling with observability — every 5xx exception is automatically logged with full context and raises an alert. You also distinguish operational errors (network timeout, DB unavailable) from programmer errors (null ref, index out of bounds) and handle them differently in your resilience strategy.


⑩ Resilience with Polly

What it is: Making your service survive when downstream dependencies fail — retries, circuit breakers, timeouts, and fallbacks.

Junior: Understand that in a distributed system, everything eventually fails. Your code must assume the network is unreliable.

Mid-level: .NET 8 integrates Polly natively via Microsoft.Extensions.Http.Resilience.

builder.Services.AddHttpClient<PaymentClient>()
    .AddStandardResilienceHandler(opts => {
        opts.Retry.MaxRetryAttempts = 3;
        opts.Retry.Delay = TimeSpan.FromSeconds(1);
        opts.Retry.BackoffType = DelayBackoffType.Exponential;
        opts.CircuitBreaker.SamplingDuration = TimeSpan.FromSeconds(30);
        opts.CircuitBreaker.FailureRatio = 0.5; // open after 50% failures
        opts.Timeout.Timeout = TimeSpan.FromSeconds(5);
    });

Senior/Principal: You understand the circuit breaker state machine — Closed (normal) → Open (failing fast, not calling the dependency) → Half-Open (trying a test request) → Closed again. This prevents cascade failures where one slow service brings down everything that calls it. You pair retries with idempotency keys so retried payment calls don't charge a customer twice. You define per-service fallback strategies — if the recommendation service is down, return a cached default list rather than a 500. At the principal level, you design for bulkhead isolation — each downstream dependency gets its own connection pool and resilience policy so one failing dependency can't exhaust shared resources.


How it all fits together — principal engineer view

At the highest level, these 10 features compose into a single coherent production system:

  • Request arrives → Gateway applies rate limiting (③) and routes to the right API version (④)
  • Service receives request → structured log (⑤) with correlation ID, trace (②) starts
  • Feature flag check (⑧) → determines which code path runs
  • Outgoing calls → wrapped in Polly resilience (⑩) (retry + circuit breaker)
  • Data access → checks cache (⑥) first, falls back to DB
  • Exceptions → caught by global handler (⑨), return Problem Details
  • Real-time updates → pushed to browsers via SSE (⑦)
  • Health probes (①) → Kubernetes knows whether to route traffic or restart
  • Observability (②) → every metric, trace, and log lands in Grafana/Jaeger for your on-call engineer

None of these features work in isolation. A robust .NET production service has all 10 wired together from day one.


Observability

What to Log

  • Incoming requests: method, path, headers, user agent, authentication status, correlation TraceId
  • Outgoing HTTP calls: url, method, status, latency, resilience state (retry, circuit breaker)
  • Exceptions: full stack, message, classification, trace context
  • Feature flag evaluation results and cache hits/misses
  • SSE connection lifecycle events
  • Rate limiting rejections with client IP and reason

Structured log example (JSON):

{
  "Timestamp": "2024-06-01T12:34:56Z",
  "Level": "Information",
  "Message": "Order placed",
  "OrderId": 1234,
  "UserId": "user-456",
  "TraceId": "00-abcd1234ef567890abcd1234ef567890-01",
  "ElapsedMilliseconds": 250
}

Key Metrics (RED and USE methods)

Metric TypeMetricDescription
Request Rate (R)requests_per_secondNumber of HTTP requests received per sec
Error Rate (E)errors_per_secondNumber of server errors (5xx) per second
Duration (D)http_request_duration_secondsHistogram of per-request latency
Utilization (U)CPU / Memory / Network usage% or absolute resource usage
Saturation (S)Thread pool starvation, queue lengthIndicates resource exhaustion or contention

Distributed Tracing Snippet (ASP.NET Core)

// In Program.cs or Startup.cs
builder.Services.AddOpenTelemetryTracing(tracerProviderBuilder =>
{
    tracerProviderBuilder
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddJaegerExporter(o =>
        {
            o.AgentHost = "jaeger-agent";
            o.AgentPort = 6831;
        });
});

SLI / SLO Examples

SLISLO ThresholdDescription
HTTP 99th percentile latency< 300ms99% of requests complete below 300ms
Error rate< 0.1%Less than 1 error per 1000 requests
Availability> 99.9%Percentage of successful health checks

Alerts Worth Paging On

  • Spike in 5xx errors exceeding 0.5% of total requests sustained for 5 minutes
  • Liveness probe failures for a pod sustained over 1 minute
  • Rate limiter rejections spike by 3x baseline
  • Circuit breaker stays open > 30 seconds on a critical downstream
  • High saturation with thread pool exhaustion or queue buildup

Additional Realistic Code Example

Distributed Rate Limiting With Redis

builder.Services.AddRateLimiter(options =>
{
    options.AddPolicy<string, Lease>(HttpContext =>
        RateLimitPartition.GetTokenBucketLimiter(
            HttpContext.User.Identity?.Name ?? HttpContext.Connection.RemoteIpAddress?.ToString() ?? "anonymous",
            _ => new TokenBucketRateLimiterOptions
            {
                TokenLimit = 100,
                TokensPerPeriod = 10,
                ReplenishmentPeriod = TimeSpan.FromSeconds(1),
                QueueProcessingOrder = QueueProcessingOrder.OldestFirst,
                QueueLimit = 100
            }));
    options.RejectionStatusCode = 429;
});

app.UseRateLimiter();

app.MapGet("/api/resource", () =>
{
    return Results.Ok("Success");
}).RequireRateLimiting();

This example shows a distributed token bucket per user or IP, preventing abuse while allowing bursts.


Summary

The 10 .NET production features mapped here are essential pillars, from health checks and observability to resilience and feature flags. Integrating them cohesively via modern .NET and ASP.NET Core capabilities lets your services thrive in production. Diagrams illustrate how requests traverse the system, while code examples and trade-offs guide practical implementation.

Mastering these features paves the way from junior engineer basics to principal-level architecture mastery.


For an official and detailed reference on these features visit the respective Microsoft docs: