The Hook

Your message queue processes 100K msg/s. The LMAX Disruptor does 6 million msg/s on a single thread. It’s not magic — it’s mechanical sympathy: pre-allocated ring buffer, memory barriers instead of locks, and batch processing. Here’s the architecture.

The Thesis

Queues are the bottleneck. Locks, contention, cache misses, GC pressure. The Disruptor replaces the queue with a pre-allocated ring buffer + memory barriers + sequence barriers. It’s not a library — it’s a design pattern for mechanical sympathy.

The Science

The Queue Problem

Queue Operation	Cost
Lock acquisition	20-50ns (uncontended) → μs (contended)
Cache line bounce	40-100ns per hop
CAS retry	Exponential backoff
GC pressure	Object allocation per message
False sharing	Adjacent fields on same cache line

The Disruptor Architecture

Core Components:

1. Ring Buffer — Pre-allocated array of events (power of 2 size)
2. Sequences — Monotonically increasing counters (producer, consumers)
3. Memory Barriers — Volatile/atomic operations instead of locks
4. Sequence Barriers — Consumers wait on producer sequence
5. Event Processors — Batch consumers

// Simplified Disruptor structure
public final class RingBuffer<E> {
    private final Object[] entries;  // Pre-allocated
    private final long[] sequences;  // Producer + consumer sequences
    private final int mask;          // size - 1 (power of 2)
    
    public long next() {
        long next = sequences[PRODUCER_INDEX] + 1;
        // Wait for consumer to catch up (wrap check)
        while (next - sequences[CONSUMER_INDEX] >= capacity) {
            Thread.yield();  // or LockSupport.parkNanos(1)
        }
        sequences[PRODUCER_INDEX] = next;
        return next;
    }
    
    public E get(long sequence) {
        return (E) entries[(int) (sequence & mask)];
    }
}

Mechanical Sympathy Principles

Principle	Implementation
Pre-allocation	Zero GC during steady state
Single writer	No locks on write path
Cache line padding	@Contended / padded sequences
Batch processing	Process N events per loop
Busy spin	Thread.onSpinWait() vs park

Performance Numbers (LMAX)

Metric	Queue (ConcurrentLinkedQueue)	Disruptor
Throughput (1 thread)	300K ops/s	6M ops/s
Throughput (3 threads)	500K ops/s	20M ops/s
Latency (p99)	500μs	5μs
GC pressure	High	Zero (steady state)

The Protocol: Implement a Minimal Disruptor

Core Data Structures

# Python-inspired minimal Disruptor (for understanding)
import threading
import time
from typing import Generic, TypeVar, Callable

T = TypeVar('T')

class Sequence:
    def __init__(self, initial=0):
        self._value = initial
        self._lock = threading.Lock()
    
    def get(self) -> int:
        return self._value
    
    def set(self, value: int):
        self._value = value
    
    def increment_and_get(self) -> int:
        with self._lock:
            self._value += 1
            return self._value

class RingBuffer(Generic[T]):
    def __init__(self, capacity: int, event_factory: Callable[[], T]):
        # Power of 2
        assert (capacity & (capacity - 1)) == 0
        self.capacity = capacity
        self.mask = capacity - 1
        self.entries = [event_factory() for _ in range(capacity)]
        self.producer_sequence = Sequence(-1)
        self.consumer_sequences = []  # List[Sequence]
    
    def next(self) -> int:
        next_seq = self.producer_sequence.increment_and_get()
        # Wait for slowest consumer
        while True:
            min_consumer = min(s.get() for s in self.consumer_sequences)
            if next_seq - min_consumer < self.capacity:
                break
            time.sleep(0.000001)  # busy spin
        return next_seq
    
    def get(self, sequence: int) -> T:
        return self.entries[sequence & self.mask]
    
    def publish(self, sequence: int):
        # Memory barrier - ensure writes visible
        pass  # In Java: VarHandle.releaseFence()

class EventProcessor:
    def __init__(self, ring_buffer: RingBuffer, handler: Callable):
        self.sequence = Sequence(-1)
        ring_buffer.consumer_sequences.append(self.sequence)
        self.handler = handler
        self.running = False
    
    def run(self):
        self.running = True
        while self.running:
            next_seq = self.sequence.get() + 1
            # Wait for producer
            while self.ring_buffer.producer_sequence.get() < next_seq:
                time.sleep(0.000001)
            
            event = self.ring_buffer.get(next_seq)
            self.handler(event)
            self.sequence.set(next_seq)

Usage Pattern

# 1. Define event
class TradeEvent:
    def __init__(self):
        self.symbol = ""
        self.price = 0.0
        self.quantity = 0
        self.timestamp = 0

# 2. Create disruptor
ring_buffer = RingBuffer(1024 * 1024, TradeEvent)  # 1M events

# 2. Create handlers
def risk_check(event: TradeEvent):
    if event.quantity > 10000:
        alert_risk_team(event)

def persist(event: TradeEvent):
    db.write(event)

def audit_log(event: TradeEvent):
    audit.write(event)

# 3. Chain processors (pipeline)
risk_processor = EventProcessor(ring_buffer, risk_check)
persist_processor = EventProcessor(ring_buffer, persist)
audit_processor = EventProcessor(ring_buffer, audit_log)

# 4. Start processors
for p in [risk_processor, persist_processor, audit_processor]:
    threading.Thread(target=p.run, daemon=True).start()

# 5. Publish (single writer)
def publish_trade(symbol, price, qty):
    seq = ring_buffer.next()
    event = ring_buffer.get(seq)
    event.symbol = symbol
    event.price = price
    event.quantity = qty
    event.timestamp = time.time_ns()
    ring_buffer.publish(seq)

The Counterpoint

• Single writer: Hard to scale producers (use multiple ring buffers)
• Complexity: More code than queue.put()
• Debugging: Harder to inspect than queue
• Python GIL: Python implementation limited by GIL (use Java/Rust/Go)

Practical Takeaway

Use Disruptor when: >100K msg/s, latency <1ms p99, single writer, JVM/Go/Rust. Stick with queues when: <10K msg/s, multiple producers, Python, simplicity matters.

Start here: Read LMAX Disruptor paper. Implement minimal version in your language. Benchmark vs queue. The 100x throughput difference is real.

---

Sources: LMAX Disruptor white paper (2011); Thompson et al. “The LMAX Architecture”; Martin Fowler “The LMAX Architecture”; Disruptor GitHub; Mechanical Sympathy blog (Martin Thompson); “Memory Barriers” (Dmitry Vyukov); Herb Sutter “Lock-Free Code”.