Skip to content

C# Interview Guide: Rate Limiting in .NET 7+ — Fixed Window, Sliding Window, Token Bucket

Q21 — Rate Limiting in .NET 7+ — Fixed Window, Sliding Window, Token Bucket Level: Senior | Topic: Performance Optimization Why Rate Limiting? Rate limiting protects your API from overload — whether from abusive clients,...

Ekanatha Reddy Urivakili8 min986 words
title: >
  C# Interview Guide: Rate Limiting in .NET 7+ — Fixed Window, Sliding Window, Token Bucket
excerpt: >
  Deep dive into .NET 7's built-in rate limiting with practical implementation and design trade-offs for Fixed Window, Sliding Window, Token Bucket, and Concurrency Limiter algorithms.
tags: [csharp, rate-limiting, .net-7, performance-optimization, api-design]
seoTitle: >-
  Rate Limiting in .NET 7+: Algorithms, Implementation, and Best Practices
seoDescription: >-
  Master rate limiting in .NET 7 using Fixed Window, Sliding Window, Token Bucket, and Concurrency Limiter with hands-on examples and observability practices.
category: tech-stacks
---

## Q21 — Rate Limiting in .NET 7+ — Fixed Window, Sliding Window, Token Bucket

*Level: Senior | Topic: Performance Optimization*

### Why Rate Limiting?

Rate limiting is essential for protecting APIs from overload stemming from:

- Abuse or misconfiguration by clients (malicious or misbehaving).
- Sudden traffic spikes (e.g., flash crowds or DoS attacks).
- Overuse of scarce backend resources.

Without rate limiting, an API risks crashing, degraded response times, or increased costs due to scaling inefficiencies. In .NET 7, Microsoft introduced first-class support for rate limiting directly in ASP.NET Core, enabling developers to use well-tested, scalable rate limiting algorithms easily.

### Rate Limiting Algorithms in .NET 7

.NET 7 offers four algorithm implementations via the `Microsoft.AspNetCore.RateLimiting` namespace:

| Algorithm          | Description                                                                                   | Pros                                        | Cons                                      | Use Case                                  |
|--------------------|-----------------------------------------------------------------------------------------------|---------------------------------------------|-------------------------------------------|------------------------------------------|
| **Fixed Window**   | Counts requests in fixed time intervals (e.g., 1000 requests per minute).                      | Simple, low overhead                         | Burst spikes at window boundaries          | Simple limits, low complexity             |
| **Sliding Window** | Splits window into smaller increments, summing recent requests to smooth burst spikes.       | Prevents bursting at window edges            | More memory, slight complexity             | When smooth rate limiting is needed       |
| **Token Bucket**    | Tokens added at steady rate; requests consume tokens; allows bursts up to the bucket size.    | Allows natural bursts, smooth rate control  | Slight complexity, queuing management      | Most natural for APIs                     |
| **Concurrency Limiter** | Limits concurrent inflight requests rather than rate over time.                           | Protects expensive backend operations       | Doesn't limit rate, only concurrency       | Long-running heavyweight endpoints        |

### Algorithm Trade-offs

| Feature               | Fixed Window                 | Sliding Window                     | Token Bucket                     | Concurrency Limiter               |
|-----------------------|-----------------------------|----------------------------------|---------------------------------|---------------------------------|
| Burst control         | Poor at boundaries           | Good                              | Good                            | N/A                             |
| Memory usage          | Low                         | Medium                           | Medium                         | Low                             |
| Implementation complexity | Simple                      | Moderate                         | Moderate                       | Simple                         |
| Queuing support       | Optional                    | Optional                        | Yes                            | Yes                            |
| Typical use cases     | Basic limiting             | Smoother limits                 | API backpressure               | Protect expensive backend ops |

### Overview Architecture and Flow

```mermaid
flowchart LR
    A[Client Request] --> B{Rate Limiter}
    B -->|Under limit| C[Allow request]
    B -->|Over limit| D[Reject request 429 with Retry-After]
    C --> E[Process request]
    E --> F[Send Response]

    subgraph Rate Limiter Algorithms
        B1[Fixed Window]
        B2[Sliding Window]
        B3[Token Bucket]
        B4[Concurrency Limiter]
    end
    B --> B1 & B2 & B3 & B4

Basic request flow through rate limiting decision.

Sequence Diagram: Token Bucket Rate Limiting Request Flow

sequenceDiagram
    participant Client
    participant API Gateway
    participant RateLimiter
    participant Backend API

    Client->>API Gateway: Send request
    API Gateway->>RateLimiter: Check token availability
    alt Tokens available
        RateLimiter-->>API Gateway: Grant token
        API Gateway->>Backend API: Forward request
        Backend API-->>API Gateway: Response
        API Gateway-->>Client: Response
    else Tokens exhausted
        RateLimiter-->>API Gateway: Reject request (429)
        API Gateway-->>Client: 429 Retry-After
    end

Shows how token bucket limits and request flow interact.


Implementing Rate Limiting in ASP.NET Core 7+

Setting up Rate Limiting Services

The key point in .NET 7 is to configure rate limiting policies in Program.cs in the DI container. You define global and named policies which can be applied selectively to controllers or endpoints through attributes.

Complete Example with Multiple Policies

// Program.cs
builder.Services.AddRateLimiter(options =>
{
    // Global Fixed Window limiter per client IP
    options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: httpContext.Connection.RemoteIpAddress?.ToString() ?? "anon",
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 1000,
                Window = TimeSpan.FromMinutes(1),
                QueueProcessingOrder = QueueProcessingOrder.OldestFirst,
                QueueLimit = 0
            }
        ));

    // Named Token Bucket limiter for authenticated users (allows bursts)
    options.AddPolicy("authenticated-api", httpContext =>
        RateLimitPartition.GetTokenBucketLimiter(
            partitionKey: httpContext.User.Identity?.Name ?? "anon",
            factory: _ => new TokenBucketRateLimiterOptions
            {
                TokenLimit = 100,
                ReplenishmentPeriod = TimeSpan.FromSeconds(1),
                TokensPerPeriod = 10,
                AutoReplenishment = true,
                QueueLimit = 5
            }
        ));

    // Named Concurrency limiter for export endpoint (max 2 concurrent requests per user)
    options.AddPolicy("export", httpContext =>
        RateLimitPartition.GetConcurrencyLimiter(
            partitionKey: httpContext.User.Identity?.Name ?? "anon",
            factory: _ => new ConcurrencyLimiterOptions
            {
                PermitLimit = 2,
                QueueLimit = 1
            }
        ));

    // Return 429 with Retry-After header on rejection
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
    options.OnRejected = async (context, cancellationToken) =>
    {
        if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
        {
            context.HttpContext.Response.Headers.RetryAfter = retryAfter.TotalSeconds.ToString("0");
        }
        context.HttpContext.Response.StatusCode = StatusCodes.Status429TooManyRequests;
        await context.HttpContext.Response.WriteAsync("Rate limit exceeded. Retry later.", cancellationToken);
    };
});

var app = builder.Build();

app.UseRateLimiter();

app.MapControllers();

Notice how multiple policies can co-exist and be applied selectively.

Applying Policies on Endpoints

[ApiController]
[Route("[controller]")]
public class ExportController : ControllerBase
{
    // This endpoint uses concurrency limiter
    [HttpPost("export")]
    [EnableRateLimiting("export")]
    public async Task<IActionResult> ExportData(CancellationToken ct)
    {
        // Simulate work
        await Task.Delay(TimeSpan.FromSeconds(10), ct);
        return Ok("Export completed");
    }
}

[ApiController]
[Route("[controller]")]
public class ProductsController : ControllerBase
{
    // This endpoint uses token bucket limiter
    [HttpGet("products")]
    [EnableRateLimiting("authenticated-api")]
    public IActionResult GetProducts()
    {
        // Return products...
        return Ok(new[] { "ProductA", "ProductB", "ProductC" });
    }
}

Notice that applying policies is attribute-driven and flexible.


Deep Dive: Algorithms and Internal Behaviors

Fixed Window

  • Counts hits within a fixed time window (e.g., minute).
  • Efficient and simple.
  • Problem: boundary condition spikes — clients can send N requests near window boundary and again just after, doubling the effective limit briefly.

Sliding Window

  • Tracks counts in smaller intervals (e.g., seconds).
  • Total requests calculated by summing recent sub-window counts weighted by time.
  • Smooths spikes better than fixed window.
  • Slightly higher memory and CPU footprint.

Token Bucket

  • Tokens added steadily over time.
  • Each request consumes a token.
  • Allows bursts equal to the number of tokens in the bucket.
  • Preferred for API rate limiting.

Concurrency Limiter

  • Limits simultaneous in-flight requests rather than rate over time.
  • Ideal for long-running or resource-intensive operations (exports, reports).
  • Prevents backend overload regardless of request rate.

Observability for Rate Limiting

Proper observability allows you to understand the behavior and health of your rate limiting strategy.

What to Log

  • Rate limit hits: When requests are accepted or rejected.
  • Rejected requests: With reason, policy name, and remaining retry duration.
  • Queue lengths: If queues are used, log queue length and any request queuing/rejections.
  • Policy metrics: Number of active tokens, current concurrency counts.
  • Client details: Partition key (hashed or anonymized) to track per-client stats.

Example Structured Log on Rejection

{
  "timestamp": "2024-06-05T14:23:30Z",
  "level": "Warning",
  "event": "RateLimitRejected",
  "policyName": "authenticated-api",
  "clientId": "user123",
  "retryAfterSeconds": 5,
  "requestPath": "/products",
  "httpMethod": "GET",
  "remoteIp": "192.0.2.1"
}

Key Metrics

Focus on RED metrics (Request rate, Errors, Duration) for your API and USE metrics (Utilization, Saturation, Errors) for the backend components that rate limiting protects.

MetricDescriptionTooling/Collection
Request RateNumber of incoming requests per secondASP.NET Core Middleware, App Insights
Rate Limit RejectionsNumber of 429 responsesApplication logs, metrics
Request Duration (latency)Time to process requestsApplication Performance Monitoring
Concurrency UsageNumber of inflight requests per partitionCustom metrics via stat reporting
Queue Length (if applicable)Number waiting in rate limiter queueCustom application metrics

Distributed Tracing with OpenTelemetry

Integrate OpenTelemetry for end-to-end tracing:

// In Program.cs - Configure OpenTelemetry
builder.Services.AddOpenTelemetryTracing(tracerProviderBuilder =>
    tracerProviderBuilder
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSource("Microsoft.AspNetCore.RateLimiting")
        .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("MyApi"))
        .AddConsoleExporter());

Instrument your rate limiting logic if you implement custom metrics or want to trace acquires, waits, and rejections.


SLI / SLO Table and Pager Alerts

SLISLO TargetAlert Condition
Success Rate (2xx / total requests)> 99.9% (excluding 429)Success rate drops below 99.5% for 5m
Rate Limit Rejections< 0.5%Rejections spike unexpectedly (>2% for 5m)
Request Latency (p99)< 500msp99 latency over 1 second sustained
Backend Concurrent Requests< PermitLimit * 0.8Sustained concurrency > 80% of limit
Queue Length< QueueLimitQueue length high sustained (> QueueLimit)

Pager alert on unexpected elevated rejection rates or high latency to avoid impacting user experience.


Senior Insights and Best Practices

  • Always return a Retry-After header with 429 to guide client retry behavior—this prevents thundering herd problems.
  • When possible, partition by authenticated user ID or API key, not by IP address, to avoid penalizing users behind NAT or proxy.
  • Use the Concurrency Limiter for expensive long-running endpoints, rather than simple rate limiting.
  • Combine rate limiting with API gateway or CDN level controls for defense-in-depth.
  • Consider backpressure and graceful degradation strategies if rate limiting is consistently triggered.
  • Monitor and tune rate limiting parameters over time as usage patterns evolve.

Minimal Runnable Example

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRateLimiter(options =>
{
    options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(ctx =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: ctx.Connection.RemoteIpAddress?.ToString() ?? "anon",
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 5,
                Window = TimeSpan.FromSeconds(10)
            }));

    options.RejectionStatusCode = 429;
    options.OnRejected = async (ctx, ct) =>
    {
        if (ctx.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
            ctx.HttpContext.Response.Headers.RetryAfter = retryAfter.TotalSeconds.ToString("0");
        ctx.HttpContext.Response.StatusCode = 429;
        await ctx.HttpContext.Response.WriteAsync("Too many requests.", ct);
    };
});

var app = builder.Build();
app.UseRateLimiter();

app.MapGet("/", () => "Hello World!");

app.Run();

Notice this simple setup applies a fixed window global limiter with very low limits for testing.


Realistic Example: Token Bucket Per User with Retry-After Header

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddAuthentication("Bearer")
    .AddJwtBearer("Bearer", options => { /* JWT config omitted */ });

builder.Services.AddRateLimiter(options =>
{
    options.AddPolicy("user-based", httpContext =>
        RateLimitPartition.GetTokenBucketLimiter(
            partitionKey: httpContext.User.Identity?.Name ?? "anon",
            factory: _ => new TokenBucketRateLimiterOptions
            {
                TokenLimit = 20,
                TokensPerPeriod = 5,
                ReplenishmentPeriod = TimeSpan.FromSeconds(1),
                AutoReplenishment = true,
                QueueLimit = 5
            }));

    options.RejectionStatusCode = 429;
    options.OnRejected = async (ctx, ct) =>
    {
        if (ctx.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
            ctx.HttpContext.Response.Headers.RetryAfter = retryAfter.TotalSeconds.ToString("0");
        ctx.HttpContext.Response.StatusCode = 429;
        await ctx.HttpContext.Response.WriteAsync("Rate limit exceeded. Please retry later.", ct);
    };
});

var app = builder.Build();

app.UseAuthentication();
app.UseRateLimiter();

app.MapGet("/data", [EnableRateLimiting("user-based"), Authorize] () => "Secured data");

app.Run();

This example uses per-user token bucket limiting on an authenticated endpoint to allow natural burst behavior.


Under the Hood: Internal Mechanics of .NET 7 Rate Limiting Algorithms

Understanding the internal workings of .NET 7's rate limiting algorithms clarifies their suitability and performance characteristics.

Fixed Window Rate Limiter Implementation

The fixed-window rate limiter maintains a simple counter per partition key (e.g., per IP or API key) and a timestamp representing the start of the current window. Requests within the window increment the counter until the permitted limit is reached.

Internally:

  • The current timestamp is bucketed by window size (e.g., a start time rounded down to the nearest minute).
  • The limiter stores: (windowStart, count) per key.
  • On request:
    • If current time still in windowStart…windowStart+windowSize:
      • If count < permitLimit ➔ increment and allow.
      • Else reject.
    • Else reset to new windowStart, reset count to 1, allow request.

This straightforward approach has constant memory per partition key (few fields) but leads to boundary bursting because the limiter resets counts abruptly. The data structure is typically a ConcurrentDictionary keyed by partition.

Sliding Window Rate Limiter Implementation

The sliding window algorithm aims to smooth the spikes of fixed window by tracking request counts finely over a rolling interval.

Internally, it:

  • Divides the full window into sub-windows (e.g., 60 seconds divided into 6 sub-windows of 10 seconds each).
  • Keeps a time-stamped queue or ring buffer of counts per sub-window for each partition.
  • When a request arrives:
    • It removes expired sub-windows that fall outside the sliding window.
    • Sums counts in relevant sub-windows weighted proportionally.
    • Compares to the allowed limit.
  • The summation is a weighted sum because the last sub-window only partially overlaps the window.

The sliding window stores more state than fixed window (approximately subwindowCount x (timestamp + count)). The advantage is smoother rate limiting, minimizing bursts at edges.

Token Bucket Rate Limiter Implementation

Token bucket is conceptualized as a bucket with tokens; requests consume tokens. Tokens accumulate over time at a fixed rate.

Key internal components:

  • The bucket capacity (TokenLimit), refill rate (TokensPerPeriod per ReplenishmentPeriod), and current token count.
  • A timestamp marking the last refill.
  • On each request:
    • Tokens are replenished based on elapsed time since last update.
    • If at least one token is available, consume one token and grant access.
    • Else reject or queue if enabled.

Token replenishment is lazy — tokens are only recalculated when a request arrives. This avoids excess CPU usage in idle times.

The bucket state is stored per partition key, usually in a thread-safe dictionary with atomic counters and timestamp checks.

Concurrency Limiter Implementation

The concurrency limiter is semantically different — it tracks concurrent inflight requests instead of rate per time.

Internally:

  • Maintains an integer count of permits granted and capacity.
  • On request:
    • If currentConcurrency < permitLimit, grant permit and currentConcurrency++.
    • Else reject or queue request.
  • On request completion:
    • currentConcurrency--.

This requires integration with middleware to track start and end of request processing to decrement concurrency count exactly once per request.

What Happens During Queuing?

Each algorithm optionally supports queuing when permits are exhausted but the queue limit is not.

  • Incoming requests beyond permits are enqueued.
  • When permits are freed (for concurrency limiter) or tokens replenished (token bucket), queued requests are dequeued in configured order (OldestFirst or NewestFirst).
  • Queuing requires internal thread-safe FIFO queue or priority structure.
  • Care must be taken to avoid head-of-line blocking and to enforce queue size limits.

Queuing adds memory overhead and latency but smooths bursts without immediate rejections.


A Production Scenario: End-to-End Token Bucket Rate Limiting with Distributed Cache Backend

Problem Context

A public REST API offering data queries runs in a distributed environment across multiple servers behind a load balancer. The business requires:

  • Each authenticated user can make up to 100 requests per minute sustained.
  • Bursts of up to 25 requests allowed instantaneously.
  • Rate limits and tokens must be consistent and shared across all API nodes.
  • Queueing allowed up to 10 requests waiting per user.
  • Retry-After header must be returned on rejections.

Challenges

  • Local in-memory storage won't achieve consistency across API replicas.
  • Distributed locking and atomic counters needed for token buckets.
  • High availability and low latency data store required.
  • Proper diagnostics and observability needed across distributed nodes.

Architectural Solution

Leverage .NET 7 built-in token bucket rate limiter configured with a distributed backing store such as Redis.

Although ASP.NET Core doesn’t provide a built-in distributed token bucket, you can implement a custom RateLimiter backed by Redis using Lua scripts for atomicity.

Custom Token Bucket Implementation with Redis

Key Design Points

  • Token bucket state per user stored in Redis hash:
    • tokens field: current tokens available (float or integer).
    • last_updated field: last replenishment timestamp (Unix epoch millis).
  • Lua scripts atomically:
    • Calculate elapsed time since last_updated.
    • Add replenished tokens by (elapsed / replenishmentPeriod) * tokensPerPeriod.
    • Cap to TokenLimit.
    • If tokens > 0, consume one and return success.
    • Else, reflect backoff (retry after time to next token).
  • Use Redis TTL for cleanup of inactive keys.
  • Queueing managed by semaphore + task completions, or via additional Redis structures if fully distributed queueing desired (complex).

Sample Lua Script Sketch

local key = KEYS[1]
local token_limit = tonumber(ARGV[1])
local tokens_per_period = tonumber(ARGV[2])
local replenishment_period_ms = tonumber(ARGV[3])
local now = tonumber(ARGV[4])

local bucket = redis.call('HMGET', key, 'tokens', 'last_updated')
local tokens = tonumber(bucket[1]) or token_limit
local last_updated = tonumber(bucket[2]) or now

-- calculate elapsed milliseconds
local elapsed = math.max(0, now - last_updated)
local replenished_tokens = (elapsed / replenishment_period_ms) * tokens_per_period
tokens = math.min(token_limit, tokens + replenished_tokens)

if tokens < 1 then
  -- not enough tokens: return backoff time
  local retry_after_ms = replenishment_period_ms / tokens_per_period
  return {0, retry_after_ms}
else
  -- consume 1 token
  tokens = tokens - 1
  redis.call('HMSET', key, 'tokens', tokens, 'last_updated', now)
  redis.call('PEXPIRE', key, replenishment_period_ms * 2)
  return {1, 0}
end

Integration in .NET 7 with Custom RateLimiter

public class RedisTokenBucketRateLimiter : RateLimiter
{
    private readonly IConnectionMultiplexer _redis;
    private readonly string _scriptSHA;
    private readonly string _keyPrefix;
    private readonly int _tokenLimit;
    private readonly int _tokensPerPeriod;
    private readonly TimeSpan _replenishmentPeriod;

    public RedisTokenBucketRateLimiter(
        IConnectionMultiplexer redis,
        string keyPrefix,
        int tokenLimit,
        int tokensPerPeriod,
        TimeSpan replenishmentPeriod)
    {
        _redis = redis;
        _keyPrefix = keyPrefix;
        _tokenLimit = tokenLimit;
        _tokensPerPeriod = tokensPerPeriod;
        _replenishmentPeriod = replenishmentPeriod;

        // Load Lua script in Redis (simplified, run during init)
        var script = "..."; // Logic of above provided lua script here
        _scriptSHA = redis.GetDatabase().ScriptLoad(script);
    }

    public override RateLimitLease Acquire(CancellationToken cancellationToken = default)
    {
        // Synchronous convenience calls async method
        return AcquireAsync(cancellationToken).GetAwaiter().GetResult();
    }

    public override async ValueTask<RateLimitLease> AcquireAsync(CancellationToken cancellationToken = default)
    {
        var db = _redis.GetDatabase();

        var key = $"{_keyPrefix}:{GetPartitionKey()}"; // Your partition key logic here
        var now = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();

        var result = (RedisResult[])await db.ScriptEvaluateAsync(
            _scriptSHA,
            new RedisKey[] { key },
            new RedisValue[]
            {
                _tokenLimit,
                _tokensPerPeriod,
                (long)_replenishmentPeriod.TotalMilliseconds,
                now
            }).ConfigureAwait(false);

        bool allowed = (int)result[0] == 1;
        int retryAfterMs = (int)result[1];

        if (allowed)
        {
            return RateLimitLease.CreateSuccessful(1);
        }
        else
        {
            var retryAfter = TimeSpan.FromMilliseconds(retryAfterMs);
            return RateLimitLease.CreateRejected(
                metadata: new Dictionary<string, object> { { MetadataName.RetryAfter, retryAfter } });
        }
    }

    // Partition key logic, e.g. user id from context (stub)
    private string GetPartitionKey() => "user123"; 
}

Applying the Custom RateLimiter

Use the PartitionedRateLimiter API to integrate your custom rate limiter such that:

builder.Services.AddRateLimiter(options =>
{
    options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
    {
        var userId = httpContext.User.Identity?.Name ?? "anon";
        var limiter = new RedisTokenBucketRateLimiter(
            redis,
            "rate-limit:user",
            100,
            25,
            TimeSpan.FromMinutes(1));
        return RateLimitPartition.GetRateLimiterPartition(userId, _ => limiter);
    });

    options.RejectionStatusCode = 429;
    options.OnRejected = async (context, ct) =>
    {
        if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
            context.HttpContext.Response.Headers.RetryAfter = ((TimeSpan)retryAfter).TotalSeconds.ToString("0");
        context.HttpContext.Response.StatusCode = 429;
        await context.HttpContext.Response.WriteAsync("Rate limit exceeded - try later", ct);
    };
});

Observability Considerations

  • Log every rejection alongside retryAfter so clients and operators can understand backoff.
  • Emit metrics for Redis latency and error count.
  • Trace rate limiting operations with spans using OpenTelemetry.

Edge Cases and Failure Modes in Rate Limiting

Rate limiting is powerful but can break in unexpected ways if edge cases are overlooked.

Edge Case: Clock Skew and Token Replenishment

  • The token bucket uses timestamps to calculate replenishment lazily.
  • If server clocks drift significantly, tokens can be over or under consumed.
  • Compensate with synchronized server clocks (NTP).
  • Alternatively, use the backend cache's time (Redis TIME command) inside Lua scripts for accurate timings.

Edge Case: Client IP Spoofing or Proxy Issues

  • Partitioning by IP can punish clients behind proxies or NATs sharing the same IP.
  • Prefer authenticated user or API key as partition key to avoid unfair throttling.
  • If IP basis is still required, combine with header inspection (e.g., X-Forwarded-For) carefully.

Failure Mode: Rate Limiter State Loss on Restart

  • In-memory limiters lose state during deployment or crashes.
  • This causes inconsistent rate limits and possible bursts.
  • Use external distributed storage like Redis or database to retain state.

Failure Mode: Under-Provisioned Queue Limits Leading to Rejections

  • Setting queue size too low leads to more rejections under bursts.
  • Setting queue limits too high causes increased request latency and memory pressure.
  • Tune queue limits based on SLA needs and system capacity.

Performance Bottlenecks and Hot Partitions

  • Uneven traffic leads to hot partitions (e.g., one user floods API).
  • This can cause contention especially in distributed stores.
  • Consider sharding keys further or dynamic rate limiting (adaptive limits).

Performance Considerations and Tuning

Memory and CPU Impact

AlgorithmMemory per partitionCPU per requestScale Factor
Fixed Window~ 1 counter + timeVery low, O(1)Scales easily
Sliding WindowMultiple countersMedium, O(n) where n=number of sub-windowsHigher memory + CPU
Token Bucket2 fields + timestampLow, O(1)Good when bursts allowed
Concurrency LimiterSimple semaphoreLow, O(1)Depends on concurrency

Avoid large numbers of partition keys being created simultaneously. Use expiration or eviction on entries.

Cache Backend Latency

  • Latency in distributed token bucket implementation directly causes request latency.
  • Optimize Redis client pipeline and network locality.
  • Use batching or local caches in less strict scenarios.

Queueing Delays

  • Queued requests incur increased latency.
  • Use queue limits conservatively.
  • Enforce meaningful Retry-After to avoid client-side retries magnifying queues.

Observability Addendum: Instrumenting Custom Rate Limiters

To maintain observability parity for custom or distributed rate-limiters, instrument the following:

Structured Logs

Each attempt to acquire a permit logs:

{
  "timestamp": "2024-06-05T16:00:00Z",
  "event": "RateLimitCheck",
  "policy": "tokenbucket-redis",
  "userId": "user123",
  "allowed": true,
  "tokensRemaining": 15,
  "requestPath": "/api/data",
  "remoteIp": "203.0.113.42"
}

Rejections similarly include retryAfter.

Prometheus Metrics Exposition

Expose counters/gauges for:

  • rate_limit_allow_total{policy="tokenbucket-redis"}
  • rate_limit_reject_total{policy="tokenbucket-redis"}
  • rate_limit_queue_length{policy="tokenbucket-redis"}

OpenTelemetry Spans

Annotate the rate limiter acquire call:

using var activity = MyActivitySource.StartActivity("RateLimitAcquire", ActivityKind.Internal);
activity?.SetTag("rate_limit.policy", policyName);
activity?.SetTag("rate_limit.allowed", allowed);
if (!allowed)
    activity?.SetTag("rate_limit.retry_after", retryAfter.TotalSeconds);

This allows end-to-end tracing to correlate rate limiting with API latencies and client traces.


Decision Checklist for Choosing a Rate Limiting Algorithm in .NET 7

Decision FactorFixed WindowSliding WindowToken BucketConcurrency Limiter
Need absolute simplicity and minimal overhead?YesNoNoYes
Must smooth request bursts around window edges?NoYesYesNo
Require natural bursts and token replenishment?NoNoYesNo
Endpoint has expensive or stateful operations?NoNoNoYes
Need queuing support to shape excess requests?Optional, but basicOptional, moderateYesYes
Desire distributed consistent rate limiting?Possible but naive with in-memoryComplex, needs distributed stateYes, with distributed storePossible, but complex
Client identification granularity?Per IP or keyPer IP or keyPer authenticated user or keyPer user or operation
API rate limit with minimal client side latency?YesYesYesN/A

This extension adds practical depth into internal algorithm mechanics, real-world distributed implementations, edge cases, performance impacts, observability of custom solutions, and decision criteria to select the right rate limiting pattern for your .NET 7 APIs. Armed with this knowledge, senior engineers can confidently design robust and scalable rate limiting systems suited to complex production demands.

Conclusion

The introduction of native rate limiting in .NET 7 via Microsoft.AspNetCore.RateLimiting provides robust, scalable and easy-to-configure algorithms fitting different use cases:

  • Use Fixed Window for basic limits with low overhead.
  • Prefer Sliding Window for smoother spikes avoidance.
  • Use Token Bucket for API endpoints requiring burst allowance and fair throttling.
  • Protect expensive operations with Concurrency Limiter focused on concurrent requests rather than throughput.

Combined with observability best practices and thoughtful policy design, .NET 7 rate limiting can greatly enhance API robustness and reliability.


References