Skip to content

System Design Interview Questions: Overload Control and Graceful Degradation

Thirteen system design interview questions on keeping services useful under load, handling dependency failures, and recovering safely.

Ekanatha Reddy Urivakili15 min3,155 words

Most system design guides focus on building the happy path: choose a database, draw the services, and estimate traffic. Senior interviews also test what happens when the design is stressed. A dependency slows down. Retries multiply load. A queue grows without bound. A partial outage becomes a full outage.

This study guide focuses on overload control and graceful degradation: how to keep a system useful when it cannot serve every request. The questions progress from fundamentals to design-level decisions. In each answer, state your assumptions, describe the mechanism, and explain its failure modes. Naming a pattern is not enough; interviewers want to know where it sits in the request path and what happens when it activates.

Fundamentals

1. What is the difference between high availability and graceful degradation?

What the interviewer is really testing: Whether you distinguish “the system is reachable” from “the system still provides useful service.”

Model answer: High availability describes whether a system can continue serving requests despite failures. Graceful degradation describes what it does when it cannot provide its full feature set or normal performance.

For example, a shopping service might remain available for browsing and checkout while temporarily disabling personalized recommendations. That is a better outcome than letting a failing recommendation dependency block every product page.

Define the degraded mode deliberately:

  • Preserve critical operations such as authentication, payment integrity, and core reads.
  • Disable or simplify optional work.
  • Make the reduced behavior visible to users and operators.
  • Ensure fallback behavior does not violate correctness or security.

“Return stale data” is not a safe universal answer. It may be fine for a product description and unacceptable for account permissions.

Follow-up: How would you decide which features to shed first?
Hint: Rank features by user impact, correctness requirements, and dependency cost.

2. How would you estimate whether a service can handle a target request rate?

What the interviewer is really testing: Whether you can turn a traffic estimate into a capacity model, and whether you recognize the assumptions behind that model.

Model answer: Start with the workload, not a single requests-per-second number. Estimate normal and peak arrival rates, request mix, payload sizes, and expensive operations. Then estimate service time and concurrency. Little’s Law gives a useful relationship:

concurrency = arrival rate × average time in the system

If a service receives 400 requests per second and each request spends 0.2 seconds in the service, its average in-flight work is about 80 requests. That is an estimate, not a capacity guarantee: tail latency, burstiness, CPU work, connection pools, and downstream limits all matter.

Validate the model with representative load tests and production measurements. State capacity figures as assumptions until they are measured under the relevant workload.

Follow-up: Why can average latency hide a capacity problem?
Hint: Consider queueing delay and the impact of tail latency on in-flight work.

3. What is backpressure, and where would you apply it?

What the interviewer is really testing: Whether you can prevent an upstream producer from overwhelming a slower consumer.

Model answer: Backpressure is a way for a downstream component to signal that the upstream should slow down, buffer within a limit, or reject work. Without it, overload often becomes an unbounded queue, rising memory use, and eventually a crash.

Apply it at meaningful boundaries:

  • Limit concurrent work before expensive processing.
  • Bound queues and define what happens when they fill.
  • Propagate cancellation when a caller no longer needs the result.
  • Return a retryable response only when the caller can safely retry.

A bounded queue is not a complete policy. Decide whether overflow drops, rejects, blocks, or persists work, and define how that affects user-visible behavior.

Follow-up: When is blocking a producer a poor overflow policy?
Hint: Think about request threads, shared resources, and whether the producer can safely wait.

4. What is the difference between a timeout and a deadline?

What the interviewer is really testing: Whether you budget time across a request rather than giving every dependency a fresh timeout.

Model answer: A timeout limits how long one operation may wait. A deadline is the latest time by which the overall operation must finish. Propagate the deadline through the call chain so each component knows how much time remains.

If a request has a 900 ms budget, spending 800 ms waiting on the first dependency leaves little time for a second call and response serialization. A per-call timeout of 900 ms at every hop can make the overall request take much longer than the caller accepts.

Use cancellation when the deadline expires so abandoned work does not continue consuming resources. Choose timeout values from observed latency and product requirements, not by copying a framework default.

Minimal runnable example — Go: Save as main.go, then run go run main.go. It shows a bounded wait that ends when its deadline expires.

package main

import (
	"context"
	"fmt"
	"time"
)

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
	defer cancel()

	select {
	case <-time.After(500 * time.Millisecond):
		fmt.Println("work completed")
	case <-ctx.Done():
		fmt.Println("request stopped:", ctx.Err())
	}
}

Notice that the caller’s context controls how long the work may continue, rather than allowing an operation to wait indefinitely.

Follow-up: What should a service do when its inbound deadline is nearly exhausted?
Hint: Avoid starting work that cannot finish within the remaining budget.

Intermediate

5. When would you use a circuit breaker, and what does it not solve?

What the interviewer is really testing: Whether you understand the breaker as a way to stop repeated calls to an unhealthy dependency, not as a substitute for capacity planning.

Model answer: A circuit breaker observes failures or slow calls and temporarily rejects calls after a threshold. It usually has three states: closed, open, and half-open. In the half-open state it permits a limited probe to check whether the dependency has recovered.

A breaker can protect resources from waiting on a dependency that is already failing. It does not fix the dependency, guarantee a correct fallback, or prevent overload from unrelated traffic. Poor thresholds can also turn a brief failure into a prolonged outage, or allow too much traffic through during recovery.

Pair the breaker with bounded timeouts, sensible retry rules, metrics, and a deliberate fallback. Scope it narrowly enough that one unhealthy dependency or operation does not disable unrelated work.

Follow-up: What signal would you use to decide that the breaker should open?
Hint: Compare consecutive failures with a rolling error or slow-call window.

6. How do retries cause or worsen an outage?

What the interviewer is really testing: Whether you see retries as additional load, and whether you can make them safe.

Model answer: A retry repeats work when the original attempt may have failed, timed out, or succeeded without the caller receiving the result. During an outage, many clients retry together, multiplying load on an already unhealthy service.

A responsible retry policy should:

  • Retry only failures that may be transient.
  • Use a small bounded attempt count and an overall deadline.
  • Add exponential backoff with jitter to spread attempts over time.
  • Respect idempotency for operations that change state.
  • Avoid retrying independently at every layer.

For a payment request, use an idempotency key and a durable way to associate that key with the result. A timeout does not prove the payment was not processed.

Follow-up: Why is jitter useful in an exponential backoff policy?
Hint: Consider what happens when many clients share the same schedule.

7. How do you choose between a queue and synchronous processing?

What the interviewer is really testing: Whether you can trade immediate response and simple failure semantics against buffering and asynchronous completion.

Model answer: Use synchronous processing when the caller needs an immediate result and the work has a predictable duration. Use a queue when the work can complete later, producers and consumers need to be decoupled, or temporary bursts should be absorbed.

A queue moves pressure; it does not remove it. Bound queue growth, monitor age as well as depth, define delivery and deduplication behavior, and decide how to handle poison messages. A queue that accepts work faster than consumers can complete it turns overload into growing delay and storage use.

Be explicit about the contract: does the API mean “accepted for processing” or “completed successfully”? Those are different promises.

Follow-up: Which queue signal may reveal a user-impacting problem earlier than queue depth?
Hint: Measure how long the oldest message has waited.

8. How would you protect a shared dependency from a noisy tenant?

What the interviewer is really testing: Whether you can isolate resource use without harming fairness or tenant security.

Model answer: Apply limits at the boundary where tenant identity is authenticated and trusted. Options include per-tenant request quotas, concurrency limits, workload classes, and separate pools for particularly expensive work.

A single global limit protects the service but lets one tenant consume most of its capacity. Per-tenant limits improve fairness, but need careful treatment of legitimate bursts and background jobs. Use a separate global ceiling too: tenant quotas cannot protect the service if the sum of allowed quotas exceeds its safe capacity.

Do not use an untrusted request field as the tenant identity. Enforce the limit consistently across instances, or document that a local limit is an approximate guardrail rather than a global quota.

Follow-up: How would you prevent tenant limits from becoming a high-cardinality storage problem?
Hint: Consider where limits are stored and which tenants need individual tracking.

9. How should a service behave when a downstream dependency is slow but not failing?

What the interviewer is really testing: Whether you recognize that latency can consume capacity before errors become visible.

Model answer: Slow calls hold connections, worker slots, and memory for longer. That increases in-flight work and may trigger a queueing cascade. Set a deadline for the dependency call, cap concurrency, and avoid starting optional downstream work when the remaining request budget is too small.

Then choose a product-appropriate response: return cached data, omit a nonessential section, or fail quickly with a clear status. A fallback must have its own capacity and correctness analysis. Falling back to the same overloaded database through a second code path is not isolation.

Follow-up: What would you inspect to tell slow dependency calls from local queueing?
Hint: Compare per-dependency latency with local request wait time and resource saturation.

Senior and system-design level

10. Design overload protection for an API with optional recommendations.

What the interviewer is really testing: Whether you can separate critical from optional work and describe the behavior of the whole request path under stress.

Model answer: Start by confirming the product contract: the core product response must succeed without recommendations, and the response can identify when recommendations were omitted. Give the recommendation call its own deadline and concurrency budget; do not let it consume all request workers.

A simple design is to fetch core data first, then request recommendations within the remaining budget. If the call times out, the breaker opens, or the optional-work budget is exhausted, return the core response. Cache recommendations only if their freshness and authorization rules permit it.

flowchart LR
    Client["Client request"] --> API["API admission"]
    API --> Core["Core product data"]
    API --> Rec["Recommendation call"]
    Rec --> Decision["Deadline and breaker"]
    Decision --> Response["Response with optional section"]
    Core --> Response

Caption: Keep optional recommendation work from determining whether the core response can succeed.

Track omission and failure separately. If recommendations are silently missing, operators cannot distinguish a deliberate shed from a broken integration.

Follow-up: What if recommendation results are personalized and authorization-sensitive?
Hint: Treat cache keys, identity, and fallback behavior as part of the security boundary.

11. How do you decide which work to shed during overload?

What the interviewer is really testing: Whether your degradation policy follows business priorities rather than technical convenience.

Model answer: Classify work by its value and consequences. Preserve operations whose failure would corrupt state, violate a contract, or block a critical user journey. Shed optional enrichment, expensive analytics, or non-urgent background work first, if the product permits it.

Implement shedding at admission points and expensive dependency boundaries. Keep limits explicit: for example, prioritize authenticated interactive traffic over a lower-priority batch class, while retaining a global ceiling for safety. Avoid uncontrolled priority schemes that starve lower-priority work indefinitely.

Agree on fallback semantics with product and security stakeholders. “Serve stale” needs a freshness bound and must not bypass authorization; “queue it for later” needs a clear acceptance contract.

Follow-up: How would you detect that a low-priority class is being starved?
Hint: Compare waiting time and completion rate by workload class.

12. How would you design recovery after a dependency outage?

What the interviewer is really testing: Whether your design can return to normal without causing a second failure during recovery.

Model answer: Recovery is a load event. Once a dependency is healthy, a large backlog or a burst of half-open probes can overwhelm it again. Limit probes, ramp traffic gradually, and drain queued work at a rate the dependency can sustain.

Protect writes against duplicate processing and make replay behavior explicit. If a queue was paused, check message age, retry counts, and poison-message handling before releasing the whole backlog. Confirm that caches and replicas have converged where correctness requires it.

A good recovery plan has observable stages: dependency health confirmed, limited traffic restored, backlog draining within bounds, and normal service verified.

Follow-up: What should happen if the dependency passes a probe but fails under real traffic?
Hint: Keep ramp-up reversible and retain the ability to shed or route away work.

13. How would you model capacity when multiple services share a database?

What the interviewer is really testing: Whether you reason about a shared bottleneck instead of treating each service’s capacity independently.

Model answer: Treat the database as a shared resource with a finite connection, CPU, I/O, and lock budget. Estimate the query mix from all callers, including background jobs, and measure the database under representative concurrent load. A service may appear healthy until its combined demand saturates the shared dependency.

Use connection pooling and per-caller concurrency limits so the application fleet cannot create an unbounded number of database operations. Separate workloads or use read replicas only when their consistency and operational costs fit the requirements. Caching can reduce reads, but introduces invalidation, staleness, and cache-stampede risks.

Set budgets from measured behavior and retain headroom for bursts and maintenance. Do not assume that adding application instances increases database capacity.

Follow-up: What failure mode can appear if every instance has a large connection pool?
Hint: Multiply the per-instance maximum by the number of instances.

Comparing overload-control choices

Use these mechanisms together only when their responsibilities are clear. The right choice depends on whether the problem is excess arrival rate, slow work, or an unhealthy dependency.

MechanismBest fitMain trade-offCommon failure
Bounded concurrencyLimit simultaneous expensive workSome requests wait or are rejectedLimit is set too high to protect the resource
Bounded queueSmooth short burstsAdds delay and storage pressureQueue grows without an age or size policy
Circuit breakerStop calling a failing dependencyMay reject calls that could have succeededThresholds or recovery probes are poorly chosen
Retry with backoffRecover from transient failuresAdds attempts and latencyRetry amplification during an outage
Rate limitEnforce an arrival budgetSome valid bursts are rejectedLimit is not tenant-aware or globally enforced

14. How would you observe whether overload controls are working?

What the interviewer is really testing: Whether you can connect control behavior to user impact, rather than treating a low error count as proof of health.

Model answer: Instrument both the service and the resources it depends on. For the service, use RED: request rate, errors, and duration, with latency distributions rather than averages alone. For resources, use USE: utilization, saturation, and errors. Measure rejected or shed requests, queue age, in-flight work, dependency timeouts, and breaker state.

Include request or trace correlation identifiers in structured logs, but do not log credentials, tokens, or sensitive payloads. Keep labels bounded: tenant IDs and arbitrary URL paths can create high-cardinality metrics and cost.

Create SLOs around user-visible outcomes. A service might have an availability objective for core requests and a separate objective for a noncritical feature. Page when the user impact or sustained saturation warrants intervention, not on every transient control activation.

Structured log example:

{
  "event": "dependency_call",
  "service": "catalog-api",
  "dependency": "recommendations",
  "outcome": "shed",
  "reason": "deadline_budget_exhausted",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "duration_ms": 42
}

Distributed tracing with OpenTelemetry (Go): Create spans around dependency work and record the outcome that explains why work was skipped or failed. Configure the SDK and exporter in application startup.

import (
	"context"

	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/attribute"
	"go.opentelemetry.io/otel/codes"
)

func loadRecommendations(ctx context.Context) error {
	ctx, span := otel.Tracer("catalog-api").Start(ctx, "recommendations.load")
	defer span.End()

	err := callRecommendationService(ctx)
	if err != nil {
		span.RecordError(err)
		span.SetStatus(codes.Error, "recommendation call failed")
		return err
	}

	span.SetAttributes(attribute.String("dependency", "recommendations"))
	return nil
}

A trace should show the dependency’s time and outcome, while metrics reveal whether the pattern is widespread. Keep sensitive data out of span attributes.

ObjectiveExample SLIExample SLOPage when
Core API availabilitySuccessful eligible requests divided by eligible requestsSet a target from the product’s reliability needsError-budget burn is sustained and user-visible
Core API latencyEligible requests completed within the latency thresholdSet a threshold from the user journeyLatency burn is sustained or critical paths time out
Queue healthAccepted work completed within its promised timeSet a completion window by workload classOldest-message age threatens the completion promise

These are objective shapes, not universal targets. Alert on sustained saturation that is likely to cause user impact, rapidly growing queue age, widespread dependency timeouts, or an abnormal increase in rejected critical requests. A breaker opening for an optional feature may warrant investigation without paging immediately; a breaker blocking a critical operation may need urgent attention.

Follow-up: Why should a shed-request counter be split by workload class?
Hint: You need to distinguish intentional protection of optional work from failure of a critical path.

Common mistakes candidates make

  • Adding retries everywhere. Retries consume capacity. Explain the deadline, attempt limit, backoff, jitter, and idempotency policy.
  • Calling every cache a fallback. A cache can be stale, unavailable, or unauthorized for the current request. State the data and security guarantees.
  • Using an unbounded queue as a capacity plan. It converts overload into delay and resource growth. Bound it and define overflow behavior.
  • Treating a circuit breaker as self-healing. A breaker limits calls; it does not repair the dependency or decide the product fallback.
  • Ignoring shared bottlenecks. More app instances can increase contention on a database or external API.
  • Quoting capacity without assumptions. Explain workload, concurrency, payload, latency, and the evidence needed to validate your estimate.
  • Optimizing only for average latency. Tail latency and queue wait determine whether users experience the system as responsive.
  • Leaving degradation implicit. If behavior changes under load, specify what changes and how users and operators can tell.

Quick-revision checklist

Before finishing a system design answer, check that you can explain:

  • The peak workload assumptions and likely bottlenecks.
  • How deadlines and cancellation propagate across the request path.
  • Where concurrency and queue sizes are bounded.
  • Which failures are retried, and how duplicate side effects are prevented.
  • What the breaker protects and what fallback is safe.
  • Which work is preserved or shed under pressure.
  • How recovery avoids a second overload event.
  • Which RED and USE signals expose user impact and saturation.
  • What the SLI measures, what the SLO promises, and when an alert should page.

This article was generated with AI. #AIGenerated