The multiplier is repeated work you do not have to do
Caching is not simply “put Redis in front of the database.” It is a deliberate trade: keep a reusable result somewhere faster, then decide when that result is safe to reuse. Done well, caching reduces latency, database load, and pressure on downstream services. Done carelessly, it serves the wrong customer’s data, hides updates, or turns a dependency outage into an application outage.
The largest wins usually come from work that is both expensive and repeated: product or configuration reads, reference data, computed summaries, and remote API responses. A cache cannot make a unique request cheaper if there is no reusable result. Start by finding repeated work and measuring its cost; choose a cache only after identifying what can safely be shared.
- Cache a result, not an assumption. Identify the inputs that determine the result, including tenant, user, locale, permissions, and version where relevant.
- Define freshness before choosing a TTL. “Two minutes” is a product and consistency decision, not a universal performance setting.
- Plan for misses and outages. The uncached path must remain correct, and a cache failure must not silently corrupt the response.
- Measure usefulness. A high hit rate is not enough if hits are stale, misses are costly, or cache operations add more latency than they save.
In production, caching decisions touch many disciplines: data modeling, latency budgets, business rules, distributed system reliability, and operational observability. This article expands on those foundations with practical patterns and pitfalls.
What caching changes in the request path
A cache-aside flow checks the cache first. On a miss, the application loads from the authoritative source, stores the result, and returns it. Later requests can skip that source until the entry expires or is invalidated. The source of truth remains the database or service; the cache is a performance layer, not a second authority.
Request flow: cache-aside
flowchart TD
Client["Request"] --> Key["Build complete cache key"]
Key --> Check["Read cache"]
Check -->|"Hit"| Return["Return cached result"]
Check -->|"Miss"| Source["Load authoritative source"]
Source --> Store["Store with freshness policy"]
Store --> Return
Source -->|"Failure"| Policy["Apply failure policy"]
Policy --> Error["Fail, degrade, or use permitted stale data"]Every arrow is a behavioral choice. For example, if the source fails, returning an expired value may be acceptable for a public product description but not for an account balance. Make that distinction explicit rather than letting an infrastructure library decide it accidentally.
Latency and capacity: why hits multiply
For a simple workload, approximate average read latency as:
$$\text{average latency} ≈ \text{hit rate} × \text{hit cost} + \text{miss rate} × \text{miss cost}$$
If a cache hit costs 2 ms, a miss costs 100 ms, and 95% of requests hit, this simplified calculation gives about 6.9 ms before accounting for other work.
Those numbers are illustrative, not benchmarking—real request times include network, serialization, queueing, and contention; tail latency can behave very differently from the average.
The capacity effect can matter as much as response time. A cache that absorbs repeated reads reduces work reaching a database or remote API, leaving more of its finite capacity for misses and writes. But a cache can also create load: misses trigger source reads, refresh jobs compete for resources, and cache-key cardinality consumes memory. Measure the whole path, not just the cache operation.
Comparing Latency and Cost of Hits vs Misses
| Aspect | Cache Hit | Cache Miss |
|---|---|---|
| Latency per request | Low (single or few ms) | High (tens or hundreds ms) |
| CPU & Memory impact | Moderate (lookup and serialization) | High (database/remote call) |
| Load on dependencies | None or minimal | Full load |
| Staleness risk | Possible if TTL or invalidation lax | None (fresh data) |
| Fault tolerance | Can serve stale on source outage | Miss can escalate failures |
This table frames the decisions you will face when tuning caching behavior for your workload.
Keys, scope, and freshness are the hard parts
A cache key is part of the correctness boundary. If two requests can produce different responses, their keys must differ or the cached representation must safely contain both.
- Keep keys deterministic and canonical: normalize casing or formatting when the underlying semantics treat those forms as equivalent.
- Include tenant and authorization-relevant dimensions where needed.
- Never rely on a later layer to repair a response that was selected under the wrong key.
Freshness policies and invalidation strategies
The freshness contract expresses when a cached value is still valid and safe to use.
| Strategy | Description | Use case example | Pros | Cons |
|---|---|---|---|---|
| Time-to-live (TTL) | Expire entries after fixed duration (absolute expiration) | Cache product catalog, news feeds | Simple; bounds staleness | Staleness window may be too long |
| Sliding expiration | Extend expiration on each access, expires after idle period | User sessions, authentication tokens | Keeps active entries alive | Can allow stale data to persist forever |
| Explicit invalidation on update | Remove or update cache entry immediately after source writes | Inventory stock counts, order state | Minimizes staleness | Requires coordination; complex in distributed cache |
| Refresh-ahead (background update) | Update cache before expiry transparently to user | Critical pages with high traffic | Avoids cache stampede | Implementation complexity |
| Stale-while-revalidate | Serve stale data while refreshing in background | Read-heavy product pages | Balances availability & freshness | Potential stale data served |
- Use bounded lifetimes. Absolute expiration puts a ceiling on staleness. Sliding expiration alone can keep a popular but obsolete value alive indefinitely.
- Choose invalidation deliberately. Invalidate after a successful write when prompt visibility matters; use a TTL when bounded staleness is acceptable and simpler operations are preferable.
- Consider negative results. Caching “not found” (404, nulls) can prevent repeated expensive misses, but keep its lifetime short if entries may be created soon.
- Control expiry bursts. Many keys written together with the same expiration can expire together. Jittering lifetimes or refreshing ahead can spread the resulting source load.
- Avoid unbounded keys. User-provided search terms and high-cardinality identifiers can turn a cache into a memory sink. Apply limits and monitor entry counts and evictions.
A minimal cache-aside example
This console example uses Microsoft.Extensions.Caching.Memory. Add that package to a console project, then run the program; the loader runs once while the entry remains valid.
using Microsoft.Extensions.Caching.Memory;
using var cache = new MemoryCache(new MemoryCacheOptions());
const string key = "catalog:featured";
var first = cache.GetOrCreate(key, entry =>
{
entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(30);
return LoadFeatured();
});
var second = cache.GetOrCreate(key, entry =>
{
entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromSeconds(30);
return LoadFeatured();
});
Console.WriteLine(first);
Console.WriteLine(second);
static string LoadFeatured()
{
Console.WriteLine("Loading from the source");
return "Featured items";
}
Notice: The cache avoids repeating the loader for this key, but it does not decide whether 30 seconds of staleness is acceptable.
A realistic ASP.NET Core and EF Core read
In an application with an existing EF Core context and product entity, a scoped reader can cache a tenant-scoped projection. Register IMemoryCache with AddMemoryCache and register the reader through dependency injection. The query remains authoritative on a miss, and the projection avoids caching a tracked entity graph.
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Caching.Memory;
public sealed record ProductView(Guid Id, string Name, decimal Price);
public sealed class ProductReader
{
private readonly AppDbContext _db;
private readonly IMemoryCache _cache;
public ProductReader(AppDbContext db, IMemoryCache cache)
{
_db = db;
_cache = cache;
}
public Task<ProductView?> GetAsync(
Guid tenantId,
Guid productId,
CancellationToken cancellationToken)
{
var key = $"product:v1:{tenantId}:{productId}";
return _cache.GetOrCreateAsync(key, async entry =>
{
entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(2);
return await _db.Products
.AsNoTracking()
.Where(p => p.TenantId == tenantId && p.Id == productId)
.Select(p => new ProductView(p.Id, p.Name, p.Price))
.SingleOrDefaultAsync(cancellationToken);
});
}
public void Invalidate(Guid tenantId, Guid productId)
{
_cache.Remove($"product:v1:{tenantId}:{productId}");
}
}
Notice: The tenant is present in both the query and key, while the versioned key makes future representation changes easier to roll out safely.
This example uses process-local memory. Invalidating it affects only the current application instance. In a multi-instance deployment, another instance may still return its own entry until expiry. Use a shared cache, publish invalidation events, or accept a documented staleness window; do not mistake local removal for cluster-wide invalidation.
Concurrency and failure modes
When a popular key expires, many simultaneous requests can all observe a miss and load the same source. This is a cache stampede. It can overwhelm the database precisely when a busy key becomes cold. Some cache APIs do not guarantee that only one factory runs for a key, so verify the semantics of the library you use instead of assuming single-flight behavior.
| Failure mode | Description | Mitigation strategies | Comments |
|---|---|---|---|
| Stampede | Multiple requests trigger concurrent source loads on expiry. | Request coalescing, refresh-ahead, stale-while-revalidate | Requires library or custom locking |
| Cache unavailable | Cache service is down or unreachable. | Bypass cache, fail fast, circuit-breaker patterns | Fallback to source can overload it; balance needed |
| Stale after write | Out-of-order writes cause old values to overwrite newer ones in the cache. | Versioned keys, serialized refresh, coordinated invalidation | Avoid stale reads polluting cache |
| Memory pressure | Cache eviction due to memory limits or GC without awareness by application. | Set size limits, entry sizing, monitor evictions | Local caches must balance memory with app needs |
| Bad cache data | Serialization incompatibility or partial writes corrupt cache entries. | Explicit versioning, strict serialization | Treat failure to read as a miss if safe |
- Stampede: Use per-key request coalescing, a refresh-ahead strategy, or carefully bounded stale-while-revalidate where serving stale data is allowed. Add source-side concurrency limits as another line of defense.
- Cache unavailable: Decide whether reads bypass the cache, use a permitted fallback, or fail fast. A bypass can overload the source; pair it with limits and alerts.
- Stale after write: An in-flight read can load an old value just before a write invalidates the key, then store that old value afterward. Versioned keys, coordinated invalidation, short lifetimes, or serialized refreshes can reduce this race.
- Memory pressure: Local caches compete with the application for memory. Set size limits when appropriate, define entry sizes if using size-based eviction, and monitor eviction and GC behavior.
- Bad cache data: Serialization changes, partial writes, and incompatible deployments can make shared entries unreadable. Use explicit versioning and treat a failed cache read as a recoverable miss only when that is safe.
Keep cache failure behavior aligned with the data. For non-critical display data, a short-lived stale value may preserve availability. For authorization decisions, financial balances, inventory reservations, or other sensitive state, stale reuse can be a correctness or security defect. In those cases, prefer a fresh authoritative check or a design with explicit consistency guarantees.
How to decide whether a cache is helping
Instrument cache reads, writes, misses, errors, evictions, and load duration. Tag metrics by a bounded cache name or operation, not by raw keys, user IDs, or other high-cardinality values. Correlate these metrics with source latency and request latency; a cache hit rate by itself can hide expensive misses or failing invalidation.
Key steps for evaluation:
- Measure baseline latency and source load before adding the cache.
- Estimate reuse: how often does the same complete key recur during the intended freshness window?
- Set a freshness objective with the product or data owner, then choose expiration and invalidation to meet it.
- Load-test cold starts, popular-key expiry, and cache outages—not only the warm-hit path.
- Revisit the design when response shape, authorization rules, or deployment topology changes.
Key observability points for caching layers
| What to log | Example structured log JSON | Reason |
|---|---|---|
| Cache hit, miss, eviction | { "event": "cache_hit", "cache": "product_cache", "key": "product:v1:12345", "latency_ms": 1.2 } | Track cache efficiency and latency |
| Cache errors | { "event": "cache_error", "error": "TimeoutException", "cache": "redis", "operation": "GET" } | Detect service issues and alerts |
| Cache loader duration | { "event": "cache_load", "key": "product:123", "duration_ms": 120 } | Identify costly load operations |
Key metrics to collect (RED for services):
| Metric | Purpose |
|---|---|
| Request Rate | Volume of cache GETs/PUTs/Invalidations |
| Error Rate | Cache failures or timeouts |
| Duration | Latency of cache operations |
System resource metrics (USE method):
| Resource | Metric |
|---|---|
| CPU | Utilization |
| Memory | Usage, Eviction counts |
| Network | Throughput, error rates |
Implementing OpenTelemetry tracing for cache operations in .NET
Instrumenting cache calls with OpenTelemetry helps correlate cache behavior with overall request flows.
using OpenTelemetry.Trace;
public class CachingService
{
private readonly Tracer _tracer;
private readonly IMemoryCache _cache;
public CachingService(TracerProvider tracerProvider, IMemoryCache cache)
{
_tracer = tracerProvider.GetTracer("MyApp.Cache");
_cache = cache;
}
public T GetOrCreate<T>(string key, Func<ICacheEntry, T> factory)
{
using var span = _tracer.StartActiveSpan("cache.getOrCreate");
span.SetAttribute("cache.key", key);
if (_cache.TryGetValue<T>(key, out var value))
{
span.SetAttribute("cache.hit", true);
return value;
}
span.SetAttribute("cache.hit", false);
var result = _cache.GetOrCreate<T>(key, factory);
return result;
}
}
This example shows how to create spans for cache operations to track hits and misses with context for distributed tracing.
SLI/SLO recommendations and alerting for caches
| SLI | SLO Example | Alert Threshold | Paging Alert? |
|---|---|---|---|
| Cache hit rate | ≥ 85% hits over 5 min window | Hit rate < 80% sustained 5 min | Yes - indicates cache not effective |
| Cache error rate | ≤ 1 error per 10,000 requests | Error rate > 0.5% over 5 min | Yes - cache backend or serialization failure |
| Cache latency | ≥ 95% requests < 5 ms | Median latency > 10 ms for 5 min | No - investigate but no alert unless spikes |
| Cache evictions | Normal steady-state retention | Sudden spike in evictions | Warn - if sustained, may cause evictions |
Alerts should be integrated with source impact monitoring to correlate cache degradation with user-visible slowness or errors.
The practical rule
Caching is a performance multiplier when it removes repeated expensive work without changing the meaning of a response. The multiplier depends on more than hit rate: correct key scope, an explicit freshness contract, controlled miss behavior, and observability are all part of the design.
- Start with one measured bottleneck.
- Cache the smallest useful representation.
- Keep the uncached path correct.
- If you cannot explain what makes an entry valid and what happens when the cache is empty or unavailable, the cache is not ready for production.
With these principles and observability in place, caching transforms from a vague optimization into a reliable, manageable performance layer in your applications.
Summary Table: Key Caching Design Dimensions
| Dimension | Description | Recommended Practice |
|---|---|---|
| Keys & Scope | What dimensions affect cache correctness? | Canonical keys, tenant/user scope, API contract versioning |
| Freshness | How long is data safe to reuse? | Define TTLs, use invalidation or refresh-ahead where immediate consistency is needed |
| Failure Handling | How to handle unavailable cache or stale data? | Fail fast or fallback with controlled stale data per domain needs |
| Concurrency Control | Prevent stampedes and race conditions | Request coalescing, semaphore limits, versioned keys |
| Observability | What to measure and log? | Hits, misses, errors, latency, evictions, correlated tracing |
| Capacity | How to ensure cache resource limits are respected | Set size limits, monitor memory and network, tune eviction policies |
This comprehensive view helps architect cache solutions that are not just fast but safe and sustainable.
References
- Microsoft.Extensions.Caching.Memory Documentation
- OpenTelemetry .NET Instrumentation
- Entity Framework Core Documentation
- Caching Patterns and Strategies
This expanded article adds depth on freshness policies, failure handling, concurrency controls, observability, and practical guidance for reliable production caching — moving well beyond the initial 7-minute overview.