Skip to content

CAP Theorem: What a Network Partition Forces You to Choose

CAP is not a menu of two guarantees to select in advance; it explains why a partition forces replicated systems to trade off linearizable consistency against availability.

Ekanatha Reddy Urivakili9 min1,806 words

The theorem is about a partition, not a permanent choice

The CAP theorem describes a fundamental limitation on distributed systems that replicate data across nodes: if a network partition prevents those nodes from communicating, the system cannot guarantee both consistency and availability simultaneously for every operation during that partition. It must either reject or delay some operations, or answer without being able to ensure that all clients observe one coherent global state.

This insight is often misunderstood and oversimplified by the slogan “pick two.” That saying treats consistency, availability, and partition tolerance as three orthogonal attributes from which you pick two; CAP actually explains why during a partition, a system must choose between consistency and availability. Network partitions are inevitable in a distributed setting — they are not an optional deployment mode or a rare failure event. Thus the trade-off is unavoidable, and must be explicit in your design for robustness.

Here are the precise definitions CAP uses (see Gilbert and Lynch’s original paper for formal details):

  • Consistency (C): Also called linearizability, consistency means that all operations appear to take effect in a global total order. Reads always see the most recent committed write, enforcing a single-copy semantics for the data.

  • Availability (A): Every request received by a non-failing node eventually receives a non-error response. Returning an explicit failure or refusing the request does not count as available under CAP — the system must answer successfully.

  • Partition Tolerance (P): The system continues to function despite arbitrary message loss or delay that partitions nodes into disconnected subsets. Because network failures are unavoidable at scale, systems must tolerate partitions to be distributed and resilient.

CAP’s consistency guarantee is about operation ordering visibility globally, not every application-level invariant or database constraint. A system might maintain local correctness properties or eventual convergence without being CAP consistent. Likewise, availability in CAP is a liveness property that excludes returning failure responses.

A two-replica example

Imagine a bank balance stored at two replicas, A and B. Both currently say 100 units. Then a network partition isolates A and B from each other. Two clients make conflicting withdrawal requests concurrently: one at A requests 20 units withdrawn, the other at B requests 30.

  • Consistency-first design: Replica A refuses or delays the withdrawal if it cannot coordinate with B to confirm the operation. It waits or returns an error to maintain a single global state. This sacrifices availability during the partition.

  • Availability-first design: Replica A immediately accepts the withdrawal and replies successfully. Meanwhile, B independently accepts its withdrawal. The two replicas now differ — the system must resolve the conflicts after the partition heals, potentially losing some updates.

No retry policy can fully solve this. Retrying a failed write at a replica isolated on the other side cannot know the status of previous attempts at the first replica. Idempotency helps avoid duplicates but not divergent committed state.

How the request reaches the trade-off

The decision flow for a replicated write when coordination becomes impossible looks like this:

flowchart TD
    Request["Client request reaches replica A"] --> Check["Can A coordinate with required replicas?"]
    Check -->|Yes| Commit["Coordinate and commit consistently"]
    Check -->|No| Policy{"Policy during partition"}
    Policy -->|Consistency first| Reject["Wait or return unavailable"]
    Policy -->|Availability first| Local["Accept locally and reconcile later"]

During a partition or suspected partition, a node must decide what to do. It cannot reliably detect a partition or distinguish a failed node from a slow network, so policy drives behavior under uncertainty.

A minimal executable illustration

This minimal .NET example models the high-level CAP trade-off: if the peer replica is unreachable, either refuse the write or accept the write locally. It isolates the fundamental decision, without replication or network code.

var partitioned = true;
var replicaA = "balance:100";
var replicaB = "balance:100";

if (partitioned)
{
    Console.WriteLine("Consistency-first: write not acknowledged.");
}
else
{
    replicaA = "balance:80";
    replicaB = replicaA;
    Console.WriteLine("Write committed on both replicas.");
}

Console.WriteLine($"A={replicaA}; B={replicaB}");

Notice this models only the trade-off point: refusing a write when coordination is impossible, or accepting locally and risking divergence.

A small runnable policy model

This .NET example expands the model. It simulates two replicas, a simple string register, and two policies: consistency-first rejects writes without peer coordination, availability-first accepts writes locally. It is intentionally minimal: no concurrency or durable storage — just the policy decision in code.

enum WritePolicy
{
    ConsistencyFirst,
    AvailabilityFirst
}

sealed class TwoReplicaRegister
{
    private string _a = "open";
    private string _b = "open";

    public bool TryWrite(
        string replica,
        string value,
        bool peerReachable,
        WritePolicy policy)
    {
        if (policy == WritePolicy.ConsistencyFirst && !peerReachable)
        {
            return false;
        }

        if (replica == "A")
        {
            _a = value;
        }
        else if (replica == "B")
        {
            _b = value;
        }
        else
        {
            throw new ArgumentException("Replica must be A or B.", nameof(replica));
        }

        if (peerReachable)
        {
            _a = value;
            _b = value;
        }

        return true;
    }

    public string Read(string replica) => replica switch
    {
        "A" => _a,
        "B" => _b,
        _ => throw new ArgumentException("Replica must be A or B.", nameof(replica))
    };
}

var cp = new TwoReplicaRegister();
var cpAccepted = cp.TryWrite("A", "closed", false, WritePolicy.ConsistencyFirst);
Console.WriteLine($"Consistency-first accepted: {cpAccepted}");

var ap = new TwoReplicaRegister();
var apAccepted = ap.TryWrite("A", "closed", false, WritePolicy.AvailabilityFirst);
Console.WriteLine($"Availability-first accepted: {apAccepted}");
Console.WriteLine($"A reads {ap.Read("A")}; B reads {ap.Read("B")}");

Here the availability-first write returns success but replicas diverge; the consistency-first write rejects the isolated attempt.

What “CP” and “AP” mean in practice

Engineers often use "CP" and "AP" to describe systems or operations, but these are shorthand for behavior during a network partition, not permanent product categories.

LabelTypical BehaviorWhat It ImpliesReal-World Examples
CPRequires coordination/quorum for writes; rejects or blocks if unavailable.Correct linearizable states but potential unavailability under partition.ZooKeeper, HDFS, Consul (RAFT mode)
APAccepts writes locally without coordination; risks divergence.High availability but potential inconsistency requiring reconciliation.Cassandra, Dynamo-style stores, Riak
FlexibleOffers tunable consistency; different policies per operation or API.Allows hybrid guarantees depending on business needs.MongoDB, CosmosDB, CockroachDB
  • CP systems use leader election, majority quorums, or leases to ensure linearizability during connectivity.
  • AP systems embrace eventual consistency and conflict resolution, optimizing availability at the cost of temporary divergence.
  • Many modern databases provide selectable consistency modes, offering both strategies within a single system.

Understanding CAP requires examining exact operation contracts — what acknowledgement means, which replicas must respond, what read guarantees apply, and what happens on failover or partition healing. Never trust marketing or single-label claims.

Choosing a policy for application data

The choice between consistency-first and availability-first policies should be grounded in business semantics and data correctness needs, not on abstract “strong” vs. “weak” ideas.

  • Favor coordination for operations enforcing invariants. For example: financial debits, inventory reservations, unique constraints, ownership transfers. Here, rejecting or delaying requests during a partition avoids incompatible outcomes.

  • Favor availability for mergeable, tolerant work. Examples include draft edits, telemetry, “likes,” or user preferences where temporary divergence can be merged and resolved later.

  • Make uncertainty visible at the API boundary. For example, if a write is locally accepted but not globally committed, the API should clearly signal the potential for later reconciliation, rather than implying global durability.

  • Separate endpoints by invariant. Different operations or data may use different policies. For example, a product listing can afford stale reads, but the payment processing endpoint requires coordination.

Below is a practical decision checklist when choosing policies for replicated data:

Data Type / Use CaseConsistency PriorityAvailable StrategyRationale
Account balances, paymentsHighCP coordination with quorum or leader electionMust prevent double spending or overdraft
Inventory reservationsHighCP or transactional coordinationMust avoid overselling
User-generated content draftsMedium/LowAvailability-first with conflict resolution (last writer wins, CRDTs)Allows responsiveness, conflicts manageable
Metrics, telemetry, logsLowAP with eventual consistency, merge resetsEventual consistency suffices

Common mistakes and pitfalls

  • “We choose CA.” A single node is indeed consistent and available, but not partition-tolerant. The distributed CAP trade-offs apply once you replicate across networked nodes.

  • “Eventual consistency equals CAP availability.” Eventual consistency only promises convergence given enough time and no new writes. It does not ensure every request is answered during a partition.

  • “A timeout proves the peer is down.” Network timeouts are insufficient evidence to distinguish a partition from slow requests or transient issues. Aggressive assumptions can cause unnecessary unavailability or inconsistent failover.

  • “We will just merge later.” Conflict resolution must be explicitly defined for your data model. Naive last-write-wins might discard valid updates; sums or counters can be incorrect without commutativity guarantees.

  • “CAP covers every distributed systems trade-off.” CAP focuses on consistency and availability during partitions, but latency, durability, operational complexity, recovery time, and consistency outside partitions are also critical design concerns.

Trade-offs at a glance

The following table summarizes trade-offs between consistency-first (CP) and availability-first (AP) approaches under a partition:

FactorConsistency-first (CP)Availability-first (AP)
Availability during partitionReduced: may reject or delay requestsHigh: serves all requests locally
Consistency guaranteeStrong linearizabilityEventual consistency, may diverge
User experienceMay see errors or stalled requests during partitionsWrites appear responsive with potential conflicting states
ComplexityComplex consensus protocols (Paxos, Raft, etc.) requiredConflict resolution and merge logic needed
Data correctness riskLow during partitionHigh: risk of lost or conflicting updates
Recovery after partitionUsually automatic recovery due to coordinationRequires manual or algorithmic reconciliation
Use casesFinancial, inventory, unique constraintsSocial media, collaborative editing, telemetry

Understanding these trade-offs helps align system design with business priorities, driving explicit policy choices rather than vague labels.

Observability for CAP systems

Monitoring and observability are vital to verify that your system behaves as intended under partitions and to troubleshoot issues.

What to log

  • Operation metadata: request ID, client ID, timestamp, operation type (read/write), replica ID
  • Coordination attempts: latency and outcome of quorum or leader coordination
  • Partition detection or network errors: timeouts, failed connections, suspect peer state
  • Rejected operations (consistency-first): reason for unavailability, policy in effect
  • Locally accepted but uncoordinated writes (availability-first): conflict flags, reconciliation status
  • State transitions: leader election, quorum membership changes

Example structured JSON log from a consistency-first write attempt failing due to unavailable quorum:

{
  "timestamp": "2024-06-15T10:32:12Z",
  "level": "WARN",
  "service": "payment-api",
  "operation": "WriteBalanceUpdate",
  "replica": "A",
  "requestId": "abc123",
  "peerReachable": false,
  "policy": "ConsistencyFirst",
  "result": "RejectedUnavailable",
  "message": "Write rejected due to unavailable quorum"
}

Key metrics

Use combined RED and USE metrics to measure system health and performance under partitions:

MetricDescriptionTarget / Alert Condition
Request Rate (RED)Number of read and write requests per secondSudden drops or spikes indicating availability issues
Error Rate (RED)Proportion of failed or rejected requestsAlert if errors exceed baseline or SLA threshold
Duration / Latency (RED)Operation latency percentiles (p95, p99)Alert on increased latency indicating coordination delays
CPU/Memory (USE)Resource utilization on replicasAlert on resource exhaustion risking unresponsiveness
Network I/O (USE)Message rate and dropped packetsAlert on unusual partitions or network failures
Leader StabilityFrequency of leader changes in consensusAlert if frequent elections disrupt consistency

Distributed tracing with OpenTelemetry (.NET example)

Instrument your coordination and API layers to trace request paths and latencies. Here is an example to create a span around a write attempt:

using OpenTelemetry;
using OpenTelemetry.Trace;

var tracerProvider = Sdk.CreateTracerProviderBuilder()
    .AddSource("MyDistributedSystem")
    .AddConsoleExporter()
    .Build();

var tracer = tracerProvider.GetTracer("MyDistributedSystem");

using var span = tracer.StartActiveSpan("TryWrite");
span.SetAttribute("replica", "A");
span.SetAttribute("policy", "ConsistencyFirst");
span.SetAttribute("peerReachable", false);

bool result = TryWrite(replica: "A", value: "closed", peerReachable: false, policy: WritePolicy.ConsistencyFirst);

span.SetAttribute("result", result ? "accepted" : "rejected");
span.End();

Instrumenting calls with tracing lets you correlate failed writes with network issues or slow consensus rounds for rapid troubleshooting.

Typical SLI and SLO definitions

SLIDescriptionSLO Example
Successful Write RatePercentage of client writes accepted without error99.9% availability over rolling 30 days
Consistent ReadsPercentage of reads returning linearizable values99.99% during normal operation
Request Latency95th percentile latency for reads/writes< 150 ms for 95% of requests
Partition Recovery TimeTime to restore full coordination after partition< 5 minutes

Alerts to page on

  • High rate of request rejections due to unavailable quorum (CP systems)
  • Sustained packet loss or network partitions between replicas
  • Leader election flaps causing temporary service unavailability
  • Sudden divergence detected in availability-first systems
  • Replica unresponsiveness beyond acceptable timeout windows

Observability ensures your system enforces CAP expectations and surfaces unexpected partition consequences before impacting customers.


CAP theorem is a principle during partitions, not a fixed label. It forces us to make explicit choices about which way to bend: consistency or availability. Understanding this trade-off, quantifying it in policies, and operating systems with clear observability are essential for resilient distributed design.