Skip to content

C# Interview Guide: Span<T>, Memory<T>, and ArrayPool<T> — Zero-Allocation Patterns

Q20 — Span&lt;T&gt;, Memory&lt;T&gt;, and ArrayPool&lt;T&gt; — Zero-Allocation Patterns Level: Senior | Topic: Performance Optimization The Problem: Allocation Pressure In high-throughput .NET code, excessive heap alloca...

Ekanatha Reddy Urivakili13 min2,025 words

Q20 — Span<T>, Memory<T>, and ArrayPool<T> — Zero-Allocation Patterns

Level: Senior | Topic: Performance Optimization

The Problem: Allocation Pressure in High-Throughput .NET Code

In performance-critical .NET applications, excessive heap allocations lead to heavy garbage collector (GC) pressure. This manifests as frequent Gen 0 collections, occasional expensive Gen 2 collections, and noticeable application pauses. The latency spikes caused by GC pauses can severely degrade user experience and throughput.

These pressure points affect scenarios such as:

  • Real-time data processing and parsing
  • Network packet handling and I/O buffering
  • High-frequency trading or telemetry ingestion
  • Game loops and rendering pipelines

Zero-allocation patterns use stack memory, buffer reuse pools, and slicing views of existing arrays to reduce or eliminate heap allocations. This reduces GC work and improves throughput and latency predictability.

The .NET Types for Zero-Allocation

TypeAllocation LocationAsync-FriendlyMutabilityUse CaseLimitations
Span<T>Stack-onlyNoMutableFast slices in sync codeCan't be stored or used async
ReadOnlySpan<T>Stack-onlyNoRead-onlyParsing/input handlingSame async/store constraints
Memory<T>Heap (struct)YesMutableAsync-friendly slicesSlight overhead over Span
ReadOnlyMemory<T>Heap (struct)YesRead-onlyAsync-friendly read views
ArrayPool<T>Heap (shared pool)YesMutableBuffer reuse for large arraysMust Return rented arrays
stackallocStack (inline buffer)NoMutableFixed-size small buffersLimited to small sizes (<1KB)

Tuning your code to leverage these correctly can greatly mitigate allocation pressure.

Fundamental Concepts and Constraints

  • Stack-only vs Heap Safe: Span<T> is a ref struct and strictly stack-only. It provides the fastest access but cannot be stored on the heap or used across asynchronous method awaits. Memory<T> bridges that gap by being a regular struct that refers to memory but can be stored and passed asynchronously.
  • Buffer Lifetime: When using ArrayPool<T>, borrowed arrays must be explicitly returned. Failing to return buffers leads to implicit memory leaks: the pool retains more arrays than necessary, increasing memory footprint.
  • Stackalloc Limitations: Stack allocation is fast but limited in size by typical Stack Overflow risk. For .NET applications, practical buffer sizes are usually under 1 KiB for stack allocations.

High-Level Architecture Diagram

flowchart LR
    A[Application] --> B[Business Logic]
    B --> C[Data Parsing / Buffer Processing]
    C -->|Uses| D[Span&lt;T&gt; / ReadOnlySpan&lt;T&gt;]
    C -->|Uses| E[Memory&lt;T&gt; / ReadOnlyMemory&lt;T&gt;]
    B -->|Async Data Flow| E
    B --> F[Buffer Management]
    F --> G[ArrayPool&lt;T&gt;]
    G -->|Rent/Return| F
    F -->|Provides buffers| C
    subgraph Stack Memory
        D
    end

High-level interaction of span, memory, and array pool in .NET zero-allocation patterns.

Sequence Diagram: Processing Buffered Network Data

sequenceDiagram
    participant App as Application
    participant Pool as ArrayPool<byte>
    participant Parser as BufferParser
    participant Handler as BusinessLogic

    App ->> Pool: Rent buffer (64 KB)
    Pool -->> App: buffer[]
    App ->> Parser: Fill buffer from network stream
    Parser ->> Handler: ProcessSpan(buffer.AsSpan())
    Handler -->> Parser: Processed
    App ->> Pool: Return buffer

Sequence of renting, using, processing, and returning buffers using ArrayPool and Span.

Practical Code Examples

Below are refined, realistic examples of zero-allocation patterns applied to common scenarios in C#. Notice the interplay between performance and safety guarantees.

1. CSV Parsing Using ReadOnlySpan<char>

This minimal example demonstrates parsing a CSV row into typed fields without intermediate string allocations (beyond the final necessary string).

public static (string Name, decimal Price) ParseCsvRow(ReadOnlySpan<char> row)
{
    int commaIdx = row.IndexOf(',');
    if (commaIdx < 0)
        throw new FormatException("CSV row missing comma separator");

    ReadOnlySpan<char> namePart = row.Slice(0, commaIdx);
    ReadOnlySpan<char> pricePart = row.Slice(commaIdx + 1);

    if (!decimal.TryParse(pricePart, out decimal price))
        throw new FormatException("Invalid decimal price");

    // Allocation happens only here to create a managed string for Name.
    return (namePart.ToString(), price);
}

Notice this code avoids allocating any intermediate strings or char arrays when processing. Bound checks are inherited from Span implementations.

2. Streaming Large Files Using ArrayPool<byte>

In I/O-bound code, large buffer allocations may cause fragmentation and LOH allocations (>85KB). Renting buffers from a shared pool reduces overhead substantially.

public static async Task<int> ReadStreamChunkedAsync(Stream stream, CancellationToken ct)
{
    const int bufferSize = 64 * 1024; // 64 KiB buffer size
    byte[] buffer = ArrayPool<byte>.Shared.Rent(bufferSize);

    int totalBytesRead = 0;
    try
    {
        int bytesRead;
        while ((bytesRead = await stream.ReadAsync(buffer.AsMemory(0, bufferSize), ct)) > 0)
        {
            ProcessChunk(buffer.AsSpan(0, bytesRead)); // Zero-copy span usage
            totalBytesRead += bytesRead;
        }
        return totalBytesRead;
    }
    finally
    {
        // Returning the buffer is critical to avoid pool memory leaks
        ArrayPool<byte>.Shared.Return(buffer, clearArray: true);
    }
}

3. Async Write Using Memory<byte>

Because Span<T> cannot cross await boundaries, Memory<T> is required for async-friendly operations.

public static async Task WriteAsync(Stream destination, ReadOnlyMemory<byte> data, CancellationToken ct)
{
    await destination.WriteAsync(data, ct).ConfigureAwait(false);
}

4. Small Buffer Stack Allocation with stackalloc

For fixed-size small buffers (<1KB), stackalloc provides allocation on the stack, bypassing heap and GC.

public static string ToHexString(ReadOnlySpan<byte> bytes)
{
    Span<char> chars = stackalloc char[bytes.Length * 2];
    for (int i = 0; i < bytes.Length; i++)
    {
        _ = bytes[i].TryFormat(chars.Slice(i * 2, 2), out _, "x2");
    }
    return new string(chars);
}

This avoids any intermediate arrays or strings during conversion except for the final result.


When To Use What: Recommendations Summary

ScenarioBest ChoiceNotes
Temporary, sync, short-lived slicesSpan<T> / ReadOnlySpan<T>Fastest, zero heap allocation, stack-only
Need to persist slice beyond sync method or across async boundariesMemory<T> / ReadOnlyMemory<T>Slight overhead, can be stored or awaited
Large buffer allocations (>85KB)ArrayPool<T>Re-use buffers and avoid LOH fragmentation
Tiny fixed-size buffers (<1KB)stackallocAlways zero allocation, limited in size
Parsing textual input, no modificationReadOnlySpan<char>Efficient and safe fast text parsing

Observability for Zero-Allocation Patterns

In production or benchmarking environments focusing on zero-allocation patterns, observability is essential to verify improvements and catch regressions.

What to Log

  • Buffer rental and return events (ArrayPool), including sizes
  • Failures converting spans or buffers (e.g., parsing errors)
  • High allocation rate warnings (from counters)
  • GC events (Gen 0/1/2 collections count and pauses)
  • Async await usage crossing span boundaries (for code safety)

Example structured log

{
  "timestamp": "2024-06-01T12:00:00Z",
  "event": "ArrayPoolBufferReturned",
  "bufferSize": 65536,
  "caller": "ReadStreamChunkedAsync",
  "durationMs": 12
}

Key Metrics & Monitoring (RED + USE Model)

MetricDescriptionType
Allocation RateBytes allocated per secondRate
Gen 0/1/2 GC CountNumber of GC collectionsCounter
GC Pause DurationTime spent in GC pausesLatency
ArrayPool Rent/ReturnCounts of buffer rentals & returnsCounter
Buffer Leak WarningsCount of rented buffers not returnedCounter
Span Lifetime ViolationsErrors on span used across async awaitError Counter

Distributed Tracing Integration (OpenTelemetry)

Instrument methods manipulating buffers and spans to understand latency contributions, especially for async flows.

Example with OpenTelemetry tracing in C#:

using OpenTelemetry.Trace;

public class StreamProcessor
{
    private readonly Tracer _tracer;

    public StreamProcessor(TracerProvider tracerProvider)
    {
        _tracer = tracerProvider.GetTracer("ZeroAllocationPatterns");
    }

    public async Task<int> ReadStreamChunkedAsync(Stream stream, CancellationToken ct)
    {
        using var span = _tracer.StartActiveSpan("ReadStreamChunked", SpanKind.Internal);
        const int bufferSize = 64 * 1024;
        byte[] buffer = ArrayPool<byte>.Shared.Rent(bufferSize);

        int totalRead = 0;
        try
        {
            int read;
            while ((read = await stream.ReadAsync(buffer.AsMemory(0, bufferSize), ct)) > 0)
            {
                // Add detailed events or attributes as needed
                ProcessChunk(buffer.AsSpan(0, read));
                totalRead += read;
            }
            return totalRead;
        }
        catch(Exception ex)
        {
            span.SetStatus(Status.Error.WithDescription(ex.Message));
            throw;
        }
        finally
        {
            ArrayPool<byte>.Shared.Return(buffer, clearArray: true);
            span.End();
        }
    }

    private void ProcessChunk(Span<byte> chunk)
    {
        // Business processing
    }
}

SLIs/SLOs for Zero-Allocation Buffer Patterns

SLITargetAlert Condition
Allocation rate< 1 MB/s (assumption)Spike doubling allocation rate
Gen2 GC frequency< 1 per minuteFrequent Gen2 causing latency spikes
Buffer leak count= 0Any pooled buffer not returned
Request processing latency< 10 ms P99Latency above SLO on 95% of requests

Key Alerts Worth Paging

  • Unexpected increase in allocation rates after deployment
  • ArrayPool rentals not matched by returns for > 1 minute
  • GC pauses > 100 ms causing request latency spikes
  • Exceptions in zero-allocation parsing methods

Senior Insight and Best Practices

  • Always return ArrayPool buffers in finally blocks.

  • Avoid [Span] crossing async boundaries; use Memory<T> for async-safe references.

  • Replace string.Substring() and similar API calls with span slicing to avoid hidden allocations.

  • Benchmark with dotnet-counters targeting:

    dotnet-counters monitor --counters System.Runtime[gen-0-gc-count,gen-1-gc-count,alloc-rate]
  • Consider MemoryMarshal for advanced zero-copy struct serialization and deserialization.


By mastering the interplay between Span<T>, Memory<T>, and ArrayPool<T>, you can build .NET systems that minimize GC pressure and maximize throughput — essential skills for senior engineers driving performance optimizations in production systems.

Deep Dive: Internal Mechanics of Span and Memory

Understanding the internal implementation details of Span<T> and Memory<T> is crucial to fully leverage their zero-allocation capabilities and to avoid subtle pitfalls that lead to runtime exceptions or performance degradation.

Span<T> — Ref Struct Backed by Pointer and Length

  • Span<T> is a ref struct, which means:
    • It always lives on the stack and is never boxed or captured in closures.
    • It cannot be stored in heap-allocated fields, captured by async/iterator state machines, or assigned to object variables.
  • Internal representation:
    • Pointer/managed reference to the beginning of the data.
    • Length indicating the number of elements.
  • When you call .Slice() or other slicing methods, the new Span<T> is just a new struct pointing to an offset inside the original buffer.
  • The runtime enforces bounds checks for every access on the Span<T>. These usually get optimized away by the JIT in tight loops.

Memory<T> — Struct with Object Reference and Offset

  • Memory<T> is a normal value type struct (struct), not a ref struct.
  • Internally, it contains:
    • A reference to the underlying data buffer, typically an object (like an array).
    • An integer offset/length marking the slice boundaries.
  • Because it stores an object reference, it can be stored or captured in fields and async state machines.
  • The Memory<T> type provides a .Span property to get a Span<T>, but the .Span access is temporary and stack-bound.
  • ReadOnlyMemory<T> behaves the same way but is immutable.

How Span and Memory Relate to Underlying Data

Memory RepresentationImplications
Stack-based (Span)Fastest access, no heap allocations
Heap-based (Memory)Slightly larger struct size (~16 bytes), can be async-safe

With Span<T>, your code automatically gains zero-allocation slicing and fast access on the stack, but you must adhere to synchronous context. With Memory<T>, zero-allocation access is maintained with some flexibility at the cost of indirection.

Example: What's Actually Stored

Span<byte> span = stackalloc byte[10]; // Points to stack memory location + length=10

byte[] array = new byte[10];
Memory<byte> memory = array.AsMemory(2, 5); // Object reference to 'array', offset=2, length=5
Span<byte> extractedSpan = memory.Span;     // Stack-bound Span<byte> covering array[2..6]

Notice Memory<T> can be easily transported and awaited, but Span<T> is necessarily short-lived and synchronous.


Production Scenario: High-Performance Network Protocol Parser

Let's walk through a fully-fledged, real-world production scenario applying zero-allocation buffers, spans, and pool management for a hypothetical binary protocol parser in a server processing high throughput data streams.

Requirements

  • Process a 100 Mbps TCP stream in 8 KB chunks.
  • Parse variable-length messages prefixed by a 4-byte length header.
  • Zero or minimal heap allocations during parsing.
  • Async-friendly reading while keeping buffers rented from a pool.
  • Safe buffer reuse without data races or leaks.

Step 1: Rent a Buffer from ArrayPool

We rent a large reusable buffer for network IO to avoid LOH allocations and fragmentation.

const int bufferSize = 8 * 1024; // 8 KB chunks
byte[] buffer = ArrayPool<byte>.Shared.Rent(bufferSize);

Step 2: Async Read Loop Using Memory

We use async reads with Memory<byte> so the buffer slice can persist across await.

int bytesBuffered = 0;
int bytesRead;
while ((bytesRead = await networkStream.ReadAsync(buffer.AsMemory(bytesBuffered), cancellationToken)) > 0)
{
    bytesBuffered += bytesRead;
    
    int messagesProcessed = 0;
    var span = buffer.AsSpan(0, bytesBuffered);

    while (TryParseMessage(span, out int messageLength, out ReadOnlySpan<byte> messageBody))
    {
        ProcessMessage(messageBody);
        span = span.Slice(messageLength);
        messagesProcessed++;
    }
    bytesBuffered = span.Length;
    span.CopyTo(buffer); // Shift leftover partial message to front
}

Step 3: Zero-Allocation Message Parsing with Span

bool TryParseMessage(ReadOnlySpan<byte> buffer, out int messageLength, out ReadOnlySpan<byte> messageBody)
{
    // Need at least 4 bytes for length prefix
    if (buffer.Length < 4)
    {
        messageLength = 0;
        messageBody = default;
        return false;
    }

    // Message length encoded as big-endian int32
    messageLength = BinaryPrimitives.ReadInt32BigEndian(buffer.Slice(0, 4));
    
    if (buffer.Length < messageLength)
    {
        messageBody = default;
        return false; // not enough data yet
    }

    messageBody = buffer.Slice(4, messageLength - 4);
    return true;
}

Step 4: Processing and Pool Management Safety

  • Ensure buffer is returned in a finally block to avoid leaks.
  • Explicitly clear the rented buffer on return for security or reuse safety.
try
{
    // IO and parsing loop as above
}
finally 
{
    ArrayPool<byte>.Shared.Return(buffer, clearArray: true);
}

Why This Works

  • No intermediate allocations during the parsing loop.
  • Parsing operates on ReadOnlySpan<byte>, a stack-only lightweight view.
  • Async read operations use Memory<byte>, leaking no extra memory.
  • Buffer shifting uses Span.CopyTo, no extra allocation.
  • Pool usage avoids fragmentation and reduces GC overhead.

Edge Cases, Failure Modes, and Mitigation Strategies

Understanding failure modes is essential for reliable zero-allocation code.

1. Forgetting To Return Rented Buffers

Effect: Memory leaks in ArrayPool causing pool exhaustion or OOM.

Mitigation:

  • Always Return in finally blocks.
  • Use static analysis tools or code reviews to track all Rent calls.
  • Consider wrapping pooled arrays in IDisposable types that return buffers on Dispose.

2. Using Span<T> Beyond Its Lifetime or Across Await

Effect: Compiler errors or runtime validation exceptions if spans are escaped.

Mitigation:

  • Use Memory<T> where asynchronous flow or heap storage is needed.
  • Avoid storing Span<T> in fields, async state machines, lambda captures.
  • Static compilation enforces this, but shadow copies or unsafe code may bypass protections, so comply with best practices.

3. Oversized stackalloc Buffers Causing Stack Overflow

Effect: Stack overflow crashes or subtle runtime failures.

Mitigation:

  • Limit stackalloc sizes, typically under 1KB.
  • Profile stack usage in production.
  • For larger buffers, prefer renting from ArrayPool.

4. Partial Data or Fragmented Reads

Effect: Parsing incomplete messages causes stalls or incorrect processing.

Mitigation:

  • Buffer shifting technique to keep incomplete messages at the front.
  • Timeouts or limits on batch processing size.
  • Periodic logging or metrics for partial message counts.

5. Concurrent Access to Shared Buffers

Effect: Corruption or inconsistent data.

Mitigation:

  • Avoid sharing rented buffers across threads unless carefully synchronized.
  • Prefer buffer ownership per logical operation or thread.
  • Immutable views with ReadOnlySpan<T> reduce risk.

Performance Considerations and Microbenchmarking

Zero-allocation patterns improve throughput but require tuning:

AspectConsiderationTip
Buffer SizeToo small: more reads; Too large: more memoryTune based on workload and CPU cache size
Rent/Return OverheadSmall buffers rented frequently add overheadReuse buffers during request lifetime
Span Slicing & Bounds ChecksMinimal overhead, usually inlined by JITAvoid redundant slicing in hot paths
Stackalloc vs ArrayPoolStackalloc is faster but limited sizeUse for small buffers in tight loops
ClearArray on ReturnSecurity vs performance tradeoffDisable clear if safe and performance-critical

Minimal Benchmark Example: Span slicing vs string allocations

public static void SpanParseDemo()
{
    string csv = "name,123.45";
    ReadOnlySpan<char> rowSpan = csv.AsSpan();

    var (name, price) = ParseCsvRow(rowSpan);
    Console.WriteLine($"Name: {name}, Price: {price}");
}

Versus a baseline with string.Split(), benchmarking shows order-of-magnitude less allocation with spans.

Realistic Benchmark Scenario

  • Simulate renting buffers, reading TCP packets, and parsing messages.
  • Measure Gen 0 collections and GC pause duration.
  • Profile throughput and latency under workload.

Observability: Enhancing Diagnostics in Zero-Allocation Systems

Zero-allocation coding often shifts bottlenecks into subtle runtime behaviors. Observability must focus on:

Structured Logging Extensions

Add detailed context logs on buffer pool operations:

_logger.LogInformation("Buffer rented", new {
    BufferSize = buffer.Length,
    Caller = nameof(ReadStreamChunkedAsync),
    Timestamp = DateTime.UtcNow
});

Use log correlation with TraceId for request-scoped tracing.

Enhanced Metric Instrumentation

Instrument:

  • Buffer pool rents vs returns ratio.
  • Average rented buffer size over time.
  • Span lifetime violation counts caught by analyzers or runtime monitors.
  • GC pause time distribution with async span correlation.

Tracing Asynchronous Buffer Usage

Apply OpenTelemetry span events at:

  • Buffer acquisition
  • Each async read/write operation
  • Parsing entry and exit points
  • Buffer return and cleanup

This tracing helps correlate GC events or latency spikes to specific zero-allocation code regions.


Decision Checklist: When and How to Use Span, Memory, and ArrayPool

QuestionUse SpanUse MemoryUse ArrayPool
Is the data access strictly synchronous and short-lived?YesNoPossibly
Must the data slice escape async points or be stored on heap?NoYesPossibly
Do you control buffer lifetime and want reuse to avoid allocations?No (small slices only)NoYes
Is buffer size large (>85KB) risking LOH?NoNoYes
Buffer size small and known at compile-time?Yes, prefer stackalloc if safeNoNo
Performance critical section requiring minimal bounds check overhead?YesSlight overheadModerate overhead for renting
Need safe buffer reuse across async requests?NoNoYes
Can the buffer contents be immutable during processing?Yes (ReadOnlySpan<T> preferred)Yes (ReadOnlyMemory<T>)Yes (if only reading or not modifying)

With this deep understanding, senior engineers can craft robust, zero-allocation .NET components that are both high-throughput and maintainable, steering clear from common performance and correctness pitfalls in real-world production systems.