title: >
C# Interview Guide: Rate Limiting in .NET 7+ — Fixed Window, Sliding Window, Token Bucket
excerpt: >
Deep dive into .NET 7's built-in rate limiting with practical implementation and design trade-offs for Fixed Window, Sliding Window, Token Bucket, and Concurrency Limiter algorithms.
tags: [csharp, rate-limiting, .net-7, performance-optimization, api-design]
seoTitle: >-
Rate Limiting in .NET 7+: Algorithms, Implementation, and Best Practices
seoDescription: >-
Master rate limiting in .NET 7 using Fixed Window, Sliding Window, Token Bucket, and Concurrency Limiter with hands-on examples and observability practices.
category: tech-stacks
---
## Q21 — Rate Limiting in .NET 7+ — Fixed Window, Sliding Window, Token Bucket
*Level: Senior | Topic: Performance Optimization*
### Why Rate Limiting?
Rate limiting is essential for protecting APIs from overload stemming from:
- Abuse or misconfiguration by clients (malicious or misbehaving).
- Sudden traffic spikes (e.g., flash crowds or DoS attacks).
- Overuse of scarce backend resources.
Without rate limiting, an API risks crashing, degraded response times, or increased costs due to scaling inefficiencies. In .NET 7, Microsoft introduced first-class support for rate limiting directly in ASP.NET Core, enabling developers to use well-tested, scalable rate limiting algorithms easily.
### Rate Limiting Algorithms in .NET 7
.NET 7 offers four algorithm implementations via the `Microsoft.AspNetCore.RateLimiting` namespace:
| Algorithm | Description | Pros | Cons | Use Case |
|--------------------|-----------------------------------------------------------------------------------------------|---------------------------------------------|-------------------------------------------|------------------------------------------|
| **Fixed Window** | Counts requests in fixed time intervals (e.g., 1000 requests per minute). | Simple, low overhead | Burst spikes at window boundaries | Simple limits, low complexity |
| **Sliding Window** | Splits window into smaller increments, summing recent requests to smooth burst spikes. | Prevents bursting at window edges | More memory, slight complexity | When smooth rate limiting is needed |
| **Token Bucket** | Tokens added at steady rate; requests consume tokens; allows bursts up to the bucket size. | Allows natural bursts, smooth rate control | Slight complexity, queuing management | Most natural for APIs |
| **Concurrency Limiter** | Limits concurrent inflight requests rather than rate over time. | Protects expensive backend operations | Doesn't limit rate, only concurrency | Long-running heavyweight endpoints |
### Algorithm Trade-offs
| Feature | Fixed Window | Sliding Window | Token Bucket | Concurrency Limiter |
|-----------------------|-----------------------------|----------------------------------|---------------------------------|---------------------------------|
| Burst control | Poor at boundaries | Good | Good | N/A |
| Memory usage | Low | Medium | Medium | Low |
| Implementation complexity | Simple | Moderate | Moderate | Simple |
| Queuing support | Optional | Optional | Yes | Yes |
| Typical use cases | Basic limiting | Smoother limits | API backpressure | Protect expensive backend ops |
### Overview Architecture and Flow
```mermaid
flowchart LR
A[Client Request] --> B{Rate Limiter}
B -->|Under limit| C[Allow request]
B -->|Over limit| D[Reject request 429 with Retry-After]
C --> E[Process request]
E --> F[Send Response]
subgraph Rate Limiter Algorithms
B1[Fixed Window]
B2[Sliding Window]
B3[Token Bucket]
B4[Concurrency Limiter]
end
B --> B1 & B2 & B3 & B4
Basic request flow through rate limiting decision.
Sequence Diagram: Token Bucket Rate Limiting Request Flow
sequenceDiagram
participant Client
participant API Gateway
participant RateLimiter
participant Backend API
Client->>API Gateway: Send request
API Gateway->>RateLimiter: Check token availability
alt Tokens available
RateLimiter-->>API Gateway: Grant token
API Gateway->>Backend API: Forward request
Backend API-->>API Gateway: Response
API Gateway-->>Client: Response
else Tokens exhausted
RateLimiter-->>API Gateway: Reject request (429)
API Gateway-->>Client: 429 Retry-After
endShows how token bucket limits and request flow interact.
Implementing Rate Limiting in ASP.NET Core 7+
Setting up Rate Limiting Services
The key point in .NET 7 is to configure rate limiting policies in Program.cs in the DI container. You define global and named policies which can be applied selectively to controllers or endpoints through attributes.
Complete Example with Multiple Policies
// Program.cs
builder.Services.AddRateLimiter(options =>
{
// Global Fixed Window limiter per client IP
options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
RateLimitPartition.GetFixedWindowLimiter(
partitionKey: httpContext.Connection.RemoteIpAddress?.ToString() ?? "anon",
factory: _ => new FixedWindowRateLimiterOptions
{
PermitLimit = 1000,
Window = TimeSpan.FromMinutes(1),
QueueProcessingOrder = QueueProcessingOrder.OldestFirst,
QueueLimit = 0
}
));
// Named Token Bucket limiter for authenticated users (allows bursts)
options.AddPolicy("authenticated-api", httpContext =>
RateLimitPartition.GetTokenBucketLimiter(
partitionKey: httpContext.User.Identity?.Name ?? "anon",
factory: _ => new TokenBucketRateLimiterOptions
{
TokenLimit = 100,
ReplenishmentPeriod = TimeSpan.FromSeconds(1),
TokensPerPeriod = 10,
AutoReplenishment = true,
QueueLimit = 5
}
));
// Named Concurrency limiter for export endpoint (max 2 concurrent requests per user)
options.AddPolicy("export", httpContext =>
RateLimitPartition.GetConcurrencyLimiter(
partitionKey: httpContext.User.Identity?.Name ?? "anon",
factory: _ => new ConcurrencyLimiterOptions
{
PermitLimit = 2,
QueueLimit = 1
}
));
// Return 429 with Retry-After header on rejection
options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;
options.OnRejected = async (context, cancellationToken) =>
{
if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
{
context.HttpContext.Response.Headers.RetryAfter = retryAfter.TotalSeconds.ToString("0");
}
context.HttpContext.Response.StatusCode = StatusCodes.Status429TooManyRequests;
await context.HttpContext.Response.WriteAsync("Rate limit exceeded. Retry later.", cancellationToken);
};
});
var app = builder.Build();
app.UseRateLimiter();
app.MapControllers();
Notice how multiple policies can co-exist and be applied selectively.
Applying Policies on Endpoints
[ApiController]
[Route("[controller]")]
public class ExportController : ControllerBase
{
// This endpoint uses concurrency limiter
[HttpPost("export")]
[EnableRateLimiting("export")]
public async Task<IActionResult> ExportData(CancellationToken ct)
{
// Simulate work
await Task.Delay(TimeSpan.FromSeconds(10), ct);
return Ok("Export completed");
}
}
[ApiController]
[Route("[controller]")]
public class ProductsController : ControllerBase
{
// This endpoint uses token bucket limiter
[HttpGet("products")]
[EnableRateLimiting("authenticated-api")]
public IActionResult GetProducts()
{
// Return products...
return Ok(new[] { "ProductA", "ProductB", "ProductC" });
}
}
Notice that applying policies is attribute-driven and flexible.
Deep Dive: Algorithms and Internal Behaviors
Fixed Window
- Counts hits within a fixed time window (e.g., minute).
- Efficient and simple.
- Problem: boundary condition spikes — clients can send N requests near window boundary and again just after, doubling the effective limit briefly.
Sliding Window
- Tracks counts in smaller intervals (e.g., seconds).
- Total requests calculated by summing recent sub-window counts weighted by time.
- Smooths spikes better than fixed window.
- Slightly higher memory and CPU footprint.
Token Bucket
- Tokens added steadily over time.
- Each request consumes a token.
- Allows bursts equal to the number of tokens in the bucket.
- Preferred for API rate limiting.
Concurrency Limiter
- Limits simultaneous in-flight requests rather than rate over time.
- Ideal for long-running or resource-intensive operations (exports, reports).
- Prevents backend overload regardless of request rate.
Observability for Rate Limiting
Proper observability allows you to understand the behavior and health of your rate limiting strategy.
What to Log
- Rate limit hits: When requests are accepted or rejected.
- Rejected requests: With reason, policy name, and remaining retry duration.
- Queue lengths: If queues are used, log queue length and any request queuing/rejections.
- Policy metrics: Number of active tokens, current concurrency counts.
- Client details: Partition key (hashed or anonymized) to track per-client stats.
Example Structured Log on Rejection
{
"timestamp": "2024-06-05T14:23:30Z",
"level": "Warning",
"event": "RateLimitRejected",
"policyName": "authenticated-api",
"clientId": "user123",
"retryAfterSeconds": 5,
"requestPath": "/products",
"httpMethod": "GET",
"remoteIp": "192.0.2.1"
}
Key Metrics
Focus on RED metrics (Request rate, Errors, Duration) for your API and USE metrics (Utilization, Saturation, Errors) for the backend components that rate limiting protects.
| Metric | Description | Tooling/Collection |
|---|---|---|
| Request Rate | Number of incoming requests per second | ASP.NET Core Middleware, App Insights |
| Rate Limit Rejections | Number of 429 responses | Application logs, metrics |
| Request Duration (latency) | Time to process requests | Application Performance Monitoring |
| Concurrency Usage | Number of inflight requests per partition | Custom metrics via stat reporting |
| Queue Length (if applicable) | Number waiting in rate limiter queue | Custom application metrics |
Distributed Tracing with OpenTelemetry
Integrate OpenTelemetry for end-to-end tracing:
// In Program.cs - Configure OpenTelemetry
builder.Services.AddOpenTelemetryTracing(tracerProviderBuilder =>
tracerProviderBuilder
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation()
.AddSource("Microsoft.AspNetCore.RateLimiting")
.SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("MyApi"))
.AddConsoleExporter());
Instrument your rate limiting logic if you implement custom metrics or want to trace acquires, waits, and rejections.
SLI / SLO Table and Pager Alerts
| SLI | SLO Target | Alert Condition |
|---|---|---|
| Success Rate (2xx / total requests) | > 99.9% (excluding 429) | Success rate drops below 99.5% for 5m |
| Rate Limit Rejections | < 0.5% | Rejections spike unexpectedly (>2% for 5m) |
| Request Latency (p99) | < 500ms | p99 latency over 1 second sustained |
| Backend Concurrent Requests | < PermitLimit * 0.8 | Sustained concurrency > 80% of limit |
| Queue Length | < QueueLimit | Queue length high sustained (> QueueLimit) |
Pager alert on unexpected elevated rejection rates or high latency to avoid impacting user experience.
Senior Insights and Best Practices
- Always return a
Retry-Afterheader with 429 to guide client retry behavior—this prevents thundering herd problems. - When possible, partition by authenticated user ID or API key, not by IP address, to avoid penalizing users behind NAT or proxy.
- Use the Concurrency Limiter for expensive long-running endpoints, rather than simple rate limiting.
- Combine rate limiting with API gateway or CDN level controls for defense-in-depth.
- Consider backpressure and graceful degradation strategies if rate limiting is consistently triggered.
- Monitor and tune rate limiting parameters over time as usage patterns evolve.
Minimal Runnable Example
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddRateLimiter(options =>
{
options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(ctx =>
RateLimitPartition.GetFixedWindowLimiter(
partitionKey: ctx.Connection.RemoteIpAddress?.ToString() ?? "anon",
factory: _ => new FixedWindowRateLimiterOptions
{
PermitLimit = 5,
Window = TimeSpan.FromSeconds(10)
}));
options.RejectionStatusCode = 429;
options.OnRejected = async (ctx, ct) =>
{
if (ctx.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
ctx.HttpContext.Response.Headers.RetryAfter = retryAfter.TotalSeconds.ToString("0");
ctx.HttpContext.Response.StatusCode = 429;
await ctx.HttpContext.Response.WriteAsync("Too many requests.", ct);
};
});
var app = builder.Build();
app.UseRateLimiter();
app.MapGet("/", () => "Hello World!");
app.Run();
Notice this simple setup applies a fixed window global limiter with very low limits for testing.
Realistic Example: Token Bucket Per User with Retry-After Header
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddAuthentication("Bearer")
.AddJwtBearer("Bearer", options => { /* JWT config omitted */ });
builder.Services.AddRateLimiter(options =>
{
options.AddPolicy("user-based", httpContext =>
RateLimitPartition.GetTokenBucketLimiter(
partitionKey: httpContext.User.Identity?.Name ?? "anon",
factory: _ => new TokenBucketRateLimiterOptions
{
TokenLimit = 20,
TokensPerPeriod = 5,
ReplenishmentPeriod = TimeSpan.FromSeconds(1),
AutoReplenishment = true,
QueueLimit = 5
}));
options.RejectionStatusCode = 429;
options.OnRejected = async (ctx, ct) =>
{
if (ctx.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
ctx.HttpContext.Response.Headers.RetryAfter = retryAfter.TotalSeconds.ToString("0");
ctx.HttpContext.Response.StatusCode = 429;
await ctx.HttpContext.Response.WriteAsync("Rate limit exceeded. Please retry later.", ct);
};
});
var app = builder.Build();
app.UseAuthentication();
app.UseRateLimiter();
app.MapGet("/data", [EnableRateLimiting("user-based"), Authorize] () => "Secured data");
app.Run();
This example uses per-user token bucket limiting on an authenticated endpoint to allow natural burst behavior.
Under the Hood: Internal Mechanics of .NET 7 Rate Limiting Algorithms
Understanding the internal workings of .NET 7's rate limiting algorithms clarifies their suitability and performance characteristics.
Fixed Window Rate Limiter Implementation
The fixed-window rate limiter maintains a simple counter per partition key (e.g., per IP or API key) and a timestamp representing the start of the current window. Requests within the window increment the counter until the permitted limit is reached.
Internally:
- The current timestamp is bucketed by window size (e.g., a start time rounded down to the nearest minute).
- The limiter stores:
(windowStart, count)per key. - On request:
- If current time still in windowStart…windowStart+windowSize:
- If count < permitLimit ➔ increment and allow.
- Else reject.
- Else reset to new windowStart, reset count to 1, allow request.
- If current time still in windowStart…windowStart+windowSize:
This straightforward approach has constant memory per partition key (few fields) but leads to boundary bursting because the limiter resets counts abruptly. The data structure is typically a ConcurrentDictionary keyed by partition.
Sliding Window Rate Limiter Implementation
The sliding window algorithm aims to smooth the spikes of fixed window by tracking request counts finely over a rolling interval.
Internally, it:
- Divides the full window into sub-windows (e.g., 60 seconds divided into 6 sub-windows of 10 seconds each).
- Keeps a time-stamped queue or ring buffer of counts per sub-window for each partition.
- When a request arrives:
- It removes expired sub-windows that fall outside the sliding window.
- Sums counts in relevant sub-windows weighted proportionally.
- Compares to the allowed limit.
- The summation is a weighted sum because the last sub-window only partially overlaps the window.
The sliding window stores more state than fixed window (approximately subwindowCount x (timestamp + count)). The advantage is smoother rate limiting, minimizing bursts at edges.
Token Bucket Rate Limiter Implementation
Token bucket is conceptualized as a bucket with tokens; requests consume tokens. Tokens accumulate over time at a fixed rate.
Key internal components:
- The bucket capacity (
TokenLimit), refill rate (TokensPerPeriodperReplenishmentPeriod), and current token count. - A timestamp marking the last refill.
- On each request:
- Tokens are replenished based on elapsed time since last update.
- If at least one token is available, consume one token and grant access.
- Else reject or queue if enabled.
Token replenishment is lazy — tokens are only recalculated when a request arrives. This avoids excess CPU usage in idle times.
The bucket state is stored per partition key, usually in a thread-safe dictionary with atomic counters and timestamp checks.
Concurrency Limiter Implementation
The concurrency limiter is semantically different — it tracks concurrent inflight requests instead of rate per time.
Internally:
- Maintains an integer count of permits granted and capacity.
- On request:
- If
currentConcurrency < permitLimit, grant permit andcurrentConcurrency++. - Else reject or queue request.
- If
- On request completion:
currentConcurrency--.
This requires integration with middleware to track start and end of request processing to decrement concurrency count exactly once per request.
What Happens During Queuing?
Each algorithm optionally supports queuing when permits are exhausted but the queue limit is not.
- Incoming requests beyond permits are enqueued.
- When permits are freed (for concurrency limiter) or tokens replenished (token bucket), queued requests are dequeued in configured order (
OldestFirstorNewestFirst). - Queuing requires internal thread-safe FIFO queue or priority structure.
- Care must be taken to avoid head-of-line blocking and to enforce queue size limits.
Queuing adds memory overhead and latency but smooths bursts without immediate rejections.
A Production Scenario: End-to-End Token Bucket Rate Limiting with Distributed Cache Backend
Problem Context
A public REST API offering data queries runs in a distributed environment across multiple servers behind a load balancer. The business requires:
- Each authenticated user can make up to 100 requests per minute sustained.
- Bursts of up to 25 requests allowed instantaneously.
- Rate limits and tokens must be consistent and shared across all API nodes.
- Queueing allowed up to 10 requests waiting per user.
- Retry-After header must be returned on rejections.
Challenges
- Local in-memory storage won't achieve consistency across API replicas.
- Distributed locking and atomic counters needed for token buckets.
- High availability and low latency data store required.
- Proper diagnostics and observability needed across distributed nodes.
Architectural Solution
Leverage .NET 7 built-in token bucket rate limiter configured with a distributed backing store such as Redis.
Although ASP.NET Core doesn’t provide a built-in distributed token bucket, you can implement a custom RateLimiter backed by Redis using Lua scripts for atomicity.
Custom Token Bucket Implementation with Redis
Key Design Points
- Token bucket state per user stored in Redis hash:
tokensfield: current tokens available (float or integer).last_updatedfield: last replenishment timestamp (Unix epoch millis).
- Lua scripts atomically:
- Calculate elapsed time since
last_updated. - Add replenished tokens by
(elapsed / replenishmentPeriod) * tokensPerPeriod. - Cap to
TokenLimit. - If tokens > 0, consume one and return success.
- Else, reflect backoff (retry after time to next token).
- Calculate elapsed time since
- Use Redis TTL for cleanup of inactive keys.
- Queueing managed by semaphore + task completions, or via additional Redis structures if fully distributed queueing desired (complex).
Sample Lua Script Sketch
local key = KEYS[1]
local token_limit = tonumber(ARGV[1])
local tokens_per_period = tonumber(ARGV[2])
local replenishment_period_ms = tonumber(ARGV[3])
local now = tonumber(ARGV[4])
local bucket = redis.call('HMGET', key, 'tokens', 'last_updated')
local tokens = tonumber(bucket[1]) or token_limit
local last_updated = tonumber(bucket[2]) or now
-- calculate elapsed milliseconds
local elapsed = math.max(0, now - last_updated)
local replenished_tokens = (elapsed / replenishment_period_ms) * tokens_per_period
tokens = math.min(token_limit, tokens + replenished_tokens)
if tokens < 1 then
-- not enough tokens: return backoff time
local retry_after_ms = replenishment_period_ms / tokens_per_period
return {0, retry_after_ms}
else
-- consume 1 token
tokens = tokens - 1
redis.call('HMSET', key, 'tokens', tokens, 'last_updated', now)
redis.call('PEXPIRE', key, replenishment_period_ms * 2)
return {1, 0}
end
Integration in .NET 7 with Custom RateLimiter
public class RedisTokenBucketRateLimiter : RateLimiter
{
private readonly IConnectionMultiplexer _redis;
private readonly string _scriptSHA;
private readonly string _keyPrefix;
private readonly int _tokenLimit;
private readonly int _tokensPerPeriod;
private readonly TimeSpan _replenishmentPeriod;
public RedisTokenBucketRateLimiter(
IConnectionMultiplexer redis,
string keyPrefix,
int tokenLimit,
int tokensPerPeriod,
TimeSpan replenishmentPeriod)
{
_redis = redis;
_keyPrefix = keyPrefix;
_tokenLimit = tokenLimit;
_tokensPerPeriod = tokensPerPeriod;
_replenishmentPeriod = replenishmentPeriod;
// Load Lua script in Redis (simplified, run during init)
var script = "..."; // Logic of above provided lua script here
_scriptSHA = redis.GetDatabase().ScriptLoad(script);
}
public override RateLimitLease Acquire(CancellationToken cancellationToken = default)
{
// Synchronous convenience calls async method
return AcquireAsync(cancellationToken).GetAwaiter().GetResult();
}
public override async ValueTask<RateLimitLease> AcquireAsync(CancellationToken cancellationToken = default)
{
var db = _redis.GetDatabase();
var key = $"{_keyPrefix}:{GetPartitionKey()}"; // Your partition key logic here
var now = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
var result = (RedisResult[])await db.ScriptEvaluateAsync(
_scriptSHA,
new RedisKey[] { key },
new RedisValue[]
{
_tokenLimit,
_tokensPerPeriod,
(long)_replenishmentPeriod.TotalMilliseconds,
now
}).ConfigureAwait(false);
bool allowed = (int)result[0] == 1;
int retryAfterMs = (int)result[1];
if (allowed)
{
return RateLimitLease.CreateSuccessful(1);
}
else
{
var retryAfter = TimeSpan.FromMilliseconds(retryAfterMs);
return RateLimitLease.CreateRejected(
metadata: new Dictionary<string, object> { { MetadataName.RetryAfter, retryAfter } });
}
}
// Partition key logic, e.g. user id from context (stub)
private string GetPartitionKey() => "user123";
}
Applying the Custom RateLimiter
Use the PartitionedRateLimiter API to integrate your custom rate limiter such that:
builder.Services.AddRateLimiter(options =>
{
options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
{
var userId = httpContext.User.Identity?.Name ?? "anon";
var limiter = new RedisTokenBucketRateLimiter(
redis,
"rate-limit:user",
100,
25,
TimeSpan.FromMinutes(1));
return RateLimitPartition.GetRateLimiterPartition(userId, _ => limiter);
});
options.RejectionStatusCode = 429;
options.OnRejected = async (context, ct) =>
{
if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
context.HttpContext.Response.Headers.RetryAfter = ((TimeSpan)retryAfter).TotalSeconds.ToString("0");
context.HttpContext.Response.StatusCode = 429;
await context.HttpContext.Response.WriteAsync("Rate limit exceeded - try later", ct);
};
});
Observability Considerations
- Log every rejection alongside
retryAfterso clients and operators can understand backoff. - Emit metrics for Redis latency and error count.
- Trace rate limiting operations with spans using OpenTelemetry.
Edge Cases and Failure Modes in Rate Limiting
Rate limiting is powerful but can break in unexpected ways if edge cases are overlooked.
Edge Case: Clock Skew and Token Replenishment
- The token bucket uses timestamps to calculate replenishment lazily.
- If server clocks drift significantly, tokens can be over or under consumed.
- Compensate with synchronized server clocks (NTP).
- Alternatively, use the backend cache's time (Redis
TIMEcommand) inside Lua scripts for accurate timings.
Edge Case: Client IP Spoofing or Proxy Issues
- Partitioning by IP can punish clients behind proxies or NATs sharing the same IP.
- Prefer authenticated user or API key as partition key to avoid unfair throttling.
- If IP basis is still required, combine with header inspection (e.g., X-Forwarded-For) carefully.
Failure Mode: Rate Limiter State Loss on Restart
- In-memory limiters lose state during deployment or crashes.
- This causes inconsistent rate limits and possible bursts.
- Use external distributed storage like Redis or database to retain state.
Failure Mode: Under-Provisioned Queue Limits Leading to Rejections
- Setting queue size too low leads to more rejections under bursts.
- Setting queue limits too high causes increased request latency and memory pressure.
- Tune queue limits based on SLA needs and system capacity.
Performance Bottlenecks and Hot Partitions
- Uneven traffic leads to hot partitions (e.g., one user floods API).
- This can cause contention especially in distributed stores.
- Consider sharding keys further or dynamic rate limiting (adaptive limits).
Performance Considerations and Tuning
Memory and CPU Impact
| Algorithm | Memory per partition | CPU per request | Scale Factor |
|---|---|---|---|
| Fixed Window | ~ 1 counter + time | Very low, O(1) | Scales easily |
| Sliding Window | Multiple counters | Medium, O(n) where n=number of sub-windows | Higher memory + CPU |
| Token Bucket | 2 fields + timestamp | Low, O(1) | Good when bursts allowed |
| Concurrency Limiter | Simple semaphore | Low, O(1) | Depends on concurrency |
Avoid large numbers of partition keys being created simultaneously. Use expiration or eviction on entries.
Cache Backend Latency
- Latency in distributed token bucket implementation directly causes request latency.
- Optimize Redis client pipeline and network locality.
- Use batching or local caches in less strict scenarios.
Queueing Delays
- Queued requests incur increased latency.
- Use queue limits conservatively.
- Enforce meaningful Retry-After to avoid client-side retries magnifying queues.
Observability Addendum: Instrumenting Custom Rate Limiters
To maintain observability parity for custom or distributed rate-limiters, instrument the following:
Structured Logs
Each attempt to acquire a permit logs:
{
"timestamp": "2024-06-05T16:00:00Z",
"event": "RateLimitCheck",
"policy": "tokenbucket-redis",
"userId": "user123",
"allowed": true,
"tokensRemaining": 15,
"requestPath": "/api/data",
"remoteIp": "203.0.113.42"
}
Rejections similarly include retryAfter.
Prometheus Metrics Exposition
Expose counters/gauges for:
rate_limit_allow_total{policy="tokenbucket-redis"}rate_limit_reject_total{policy="tokenbucket-redis"}rate_limit_queue_length{policy="tokenbucket-redis"}
OpenTelemetry Spans
Annotate the rate limiter acquire call:
using var activity = MyActivitySource.StartActivity("RateLimitAcquire", ActivityKind.Internal);
activity?.SetTag("rate_limit.policy", policyName);
activity?.SetTag("rate_limit.allowed", allowed);
if (!allowed)
activity?.SetTag("rate_limit.retry_after", retryAfter.TotalSeconds);
This allows end-to-end tracing to correlate rate limiting with API latencies and client traces.
Decision Checklist for Choosing a Rate Limiting Algorithm in .NET 7
| Decision Factor | Fixed Window | Sliding Window | Token Bucket | Concurrency Limiter |
|---|---|---|---|---|
| Need absolute simplicity and minimal overhead? | Yes | No | No | Yes |
| Must smooth request bursts around window edges? | No | Yes | Yes | No |
| Require natural bursts and token replenishment? | No | No | Yes | No |
| Endpoint has expensive or stateful operations? | No | No | No | Yes |
| Need queuing support to shape excess requests? | Optional, but basic | Optional, moderate | Yes | Yes |
| Desire distributed consistent rate limiting? | Possible but naive with in-memory | Complex, needs distributed state | Yes, with distributed store | Possible, but complex |
| Client identification granularity? | Per IP or key | Per IP or key | Per authenticated user or key | Per user or operation |
| API rate limit with minimal client side latency? | Yes | Yes | Yes | N/A |
This extension adds practical depth into internal algorithm mechanics, real-world distributed implementations, edge cases, performance impacts, observability of custom solutions, and decision criteria to select the right rate limiting pattern for your .NET 7 APIs. Armed with this knowledge, senior engineers can confidently design robust and scalable rate limiting systems suited to complex production demands.
Conclusion
The introduction of native rate limiting in .NET 7 via Microsoft.AspNetCore.RateLimiting provides robust, scalable and easy-to-configure algorithms fitting different use cases:
- Use Fixed Window for basic limits with low overhead.
- Prefer Sliding Window for smoother spikes avoidance.
- Use Token Bucket for API endpoints requiring burst allowance and fair throttling.
- Protect expensive operations with Concurrency Limiter focused on concurrent requests rather than throughput.
Combined with observability best practices and thoughtful policy design, .NET 7 rate limiting can greatly enhance API robustness and reliability.