Stock Exchange (NASDAQ, NYSE)

Overview

Introduction
Requirements
Data Model
API Design
High Level Design
Deep Dive 1: The Matching Engine & Order Book
Deep Dive 2: Determinism & Event Sourcing
Deep Dive 3: Low-Latency Market Data Distribution
Deep Dive 4: Concurrency & Lock-Free Architecture
Complete Architecture
Additional Discussion Points

Introduction

A stock exchange is a centralized marketplace where financial instruments (equities, derivatives, options) are bought and sold.
Designing a modern stock exchange is fundamentally different from a typical web-scale distributed system. While web applications optimize for "good enough" latency and high availability via eventual consistency, a stock exchange prioritizes extreme low latency (measured in microseconds or nanoseconds), strict determinism, and absolute fairness. A delay of a single millisecond can cost institutional participants millions of dollars.
This is a really popular system design that interviewers at unsurprisingly High-Frequency Trading (HFT) & Hedge Funds: Citadel, Two Sigma, and Jane Street love to ask to see if you understand "mechanical sympathy" (CPU caches, lock-free data structures, kernel bypass). For them, a microsecond delay or a locked thread directly equals millions of dollars lost to a faster competitor.
This question is not asked to all levels. It is highly specific and typically reserved for Senior (L5) and Staff/Principal (L6+) engineers.

Requirements

Functional Requirements
- Order Entry: Participants can submit orders (Market, Limit) and cancel existing orders.
- Order Matching: The system must match buy and sell orders based strictly on Price-Time Priority.
- Execution Reporting: The system must notify participants of order fills, partial fills, or cancellations.
- Market Data Dissemination: The system must broadcast real-time price updates (Level 2 Order Book data) to all participants simultaneously.
- Clearing & Settlement: End-of-day processes to permanently transfer ownership and funds.
Non Functional Requirements
- Ultra-Low Latency: < 50 microseconds for the entire order matching lifecycle.
- High Throughput: Handle 1M+ messages per second, especially during market open/close bursts.
- Fairness (Strict FIFO): Orders must be sequenced and processed in the exact order they arrive at the exchange network boundary.
- Fault Tolerance: Zero data loss. If the primary matching engine crashes, a backup must take over seamlessly without losing a single order.

Data Model

To achieve microsecond latency, a stock exchange entirely abandons traditional databases in its "Hot Path" (the critical flow of order matching). It relies on highly optimized in-memory structures and append-only logs. Traditional databases are relegated to the "Cold Path" for post-trade processing.

In-Memory State (The Order Book): The active state of the market lives entirely in RAM, utilizing custom data structures (usually written in C++ or Rust) optimized for L1/L2 CPU cache lines to avoid expensive main-memory fetches.
Event Journal (Aeron / Highly Tuned Kafka): Serves as an immutable, append-only distributed log. It records every incoming order before it hits the matching engine. This is the durable "Source of Truth" used for recovery.
Relational DB (PostgreSQL / Oracle): The "Cold Path" persistence layer. Used for end-of-day (EOD) clearing, storing participant account balances, reference data (e.g., ticker symbols, tick sizes), and regulatory reporting.

API Design

Unlike web applications that use REST over HTTP, high-frequency trading systems rely on binary protocols over TCP or raw sockets to eliminate parsing overhead. The global standard is the FIX (Financial Information eXchange) protocol.

Order Entry (FIX Protocol):
- NewOrderSingle (MsgType=D): Submits a new order containing Symbol, Side (Buy/Sell), OrderQty, Price, and ClOrdID (Client Order ID).
- OrderCancelRequest (MsgType=F): Requests cancellation of an active order.
Execution Reports (FIX Protocol):
- ExecutionReport (MsgType=8): Sent back to the client acknowledging an order receipt, fill, or cancellation.
Market Data (Binary/SBE over UDP):
- Exchanges use custom binary protocols (e.g., ITCH for NASDAQ) encoded with Simple Binary Encoding (SBE) distributed via UDP Multicast to provide the lowest possible wire-size and latency.

High Level Design

At a high level, the system architecture can be separated into four core parts: order ingestion, sequential ordering, state-machine execution, and data broadcasting.

1. Order Ingestion (The Gateway Plane)

Focus: Securely receiving, parsing, and validating the order as fast as possible without touching the core matching logic.

FIX Gateway: Terminates the TCP connection from the broker, decodes the FIX message, and translates it into the exchange's internal binary format.
Pre-Trade Risk: A highly optimized, usually hardware-accelerated (FPGA) check to ensure the trader has the margin to cover the trade and isn't submitting a "fat finger" error (e.g., trying to buy 1 billion shares).

2. Sequential Ordering (The Serialization Plane)

Focus: Taking concurrent, chaotic network traffic and creating a single, perfectly ordered timeline.

The Sequencer: This is the critical bottleneck. It takes validated orders from hundreds of gateways and stamps them with a strict, monotonically increasing sequence number and a nanosecond hardware timestamp.
Event Journal: The sequenced order is written to an ultra-fast, append-only distributed log (like Aeron). This guarantees FIFO fairness and ensures zero data loss before the matching engine ever sees the order.

3. State-Machine Execution (The Compute Plane)

Focus: Processing the ordered timeline with zero concurrency issues to determine financial outcomes.

Matching Engine: A single-threaded, purely deterministic process. It reads the Event Journal sequentially, updates its internal, in-memory Order Book, and generates trade events. Because the input is strictly ordered, the engine requires no locks, no databases, and no complex concurrency control.

4. Data Broadcasting & Persistence (The Egress Plane)

Focus: Distributing the results of the execution as fast as possible to the market, while safely storing it for end-of-day settlement.

Market Data Publisher (Hot Path): Takes state changes from the matching engine, formats them into a binary payload (SBE), and blasts them out to the entire market simultaneously via UDP Multicast.
Drop Copy / Post-Trade (Cold Path): Asynchronously routes the private execution reports back to the specific trading firm via the FIX gateway, and simultaneously writes the trade to the Relational DB (PostgreSQL) for overnight clearing.

This is fine but to really stand out from other candidates we need to dive deeper into things like the order book data structures, event sourcing, thread pinning and lock-free ring buffers.

Deep Dive 1: The Matching Engine & Order Book

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.