The Hook

Your JSON parser does 200 MB/s. simdjson does 2.5 GB/s. It’s not faster algorithms — it’s SIMD: processing 64 bytes per CPU instruction. The secret: parallel bit streams that find structural characters (,, :, {, }, ") in parallel, then branchless parsing. Here’s how it works.

The Thesis

JSON parsing is memory-bandwidth bound, not CPU-bound. Standard parsers process byte-by-byte with branches. SIMD processes 64 bytes/instruction, finds all structural characters in parallel, then builds the parse tree with near-zero branches. The result: 10-15x speedup on modern CPUs.

The Science

The SIMD Advantage

Parser	Throughput	Approach
RapidJSON	~200 MB/s	Byte-by-byte, branchy
simdjson (AVX2)	~2.5 GB/s	32-byte vectors, parallel bit streams
simdjson (AVX-512)	~3.5 GB/s	64-byte vectors
simdjson (ARM NEON)	~1.8 GB/s	16-byte vectors

The Core Insight: Parallel Bit Streams

Problem: Find all structural characters ({ } [ ] : , " \) in a JSON document.

Scalar: Check each byte, branch on match.

SIMD: Load 32/64 bytes. Compare against all 7 structural chars in parallel using vector instructions. Result: 7 bitmasks (one per char class).

// AVX2: 32 bytes at a time
__m256i chunk = _mm256_loadu_si256((__m256i*)ptr);

// Compare against each structural character
__m256i is_brace_open  = _mm256_cmpeq_epi8(chunk, _mm256_set1_epi8('{'));
__m256i is_brace_close = _mm256_cmpeq_epi8(chunk, _mm256_set1_epi8('}'));
__m256i is_bracket_open = _mm256_cmpeq_epi8(chunk, _mm256_set1_epi8('['));
__m256i is_bracket_close = _mm256_cmpeq_epi8(chunk, _mm256_set1_epi8(']'));
__m256i is_colon = _mm256_cmpeq_epi8(chunk, _mm256_set1_epi8(':'));
__m256i is_comma = _mm256_cmpeq_epi8(chunk, _mm256_set1_epi8(','));
__m256i is_quote = _mm256_cmpeq_epi8(chunk, _mm256_set1_epi8('"'));

// Extract bitmasks
uint32_t brace_open_mask  = _mm256_movemask_epi8(is_brace_open);
uint32_t brace_close_mask = _mm256_movemask_epi8(is_brace_close);
// ... etc

Result: 7 uint32 bitmasks per 32 bytes. Each bit = position of that structural character.

Branchless UTF-8 Validation

Problem: Validate UTF-8 without branches.

SIMD Solution: Use lookup tables + vector compares.

// Check for valid UTF-8 continuation bytes (0x80-0xBF)
// and valid leading bytes
__m256i is_continuation = _mm256_and_si256(
    _mm256_cmpgt_epi8(chunk, _mm256_set1_epi8(0x7F)),
    _mm256_cmplt_epi8(chunk, _mm256_set1_epi8(0xC0))
);

// Check for overlong sequences, surrogates, etc.
// All done with vector compares + bitmasks

Structural Index Building

Instead of recursive descent: Build structural index from bitmasks.

struct StructuralIndex {
    uint32_t* brace_open_positions;
    uint32_t* brace_close_positions;
    uint32_t* bracket_open_positions;
    uint32_t* bracket_close_positions;
    uint32_t* colon_positions;
    uint32_t* comma_positions;
    uint32_t* string_positions;
    size_t count;
};

// Build from bitmasks using _tzcnt (trailing zero count)
void build_index(uint32_t mask, uint32_t* positions, size_t& count) {
    while (mask) {
        uint32_t tz = __builtin_ctz(mask);  // or _tzcnt_u32
        positions[count++] = base_offset + tz;
        mask &= mask - 1;  // clear lowest set bit
    }
}

Parsing from Index

Instead of recursive descent: Navigate via index arrays.

// Find matching brace: binary search in brace_close_positions
// Find object members: iterate between { and } using comma positions
// Find key-value pairs: colon positions between braces

String Parsing (The Hard Part)

Challenge: Unescape strings (\n, \t, \uXXXX, \") at SIMD speed.

Solution: Two-phase

1. Find all quotes → string boundaries (already have bitmask)
2. Vectorized unescape: Process 32 bytes at a time

// Find escape sequences
__m256i is_backslash = _mm256_cmpeq_epi8(chunk, _mm256_set1_epi8('\\'));
uint32_t escape_mask = _mm256_movemask_epi8(is_backslash);

// Process escapes in batches
// Use shuffle instructions for \n \t \r \" \\
// Unicode \uXXXX: scalar fallback (rare)

The Protocol: Build a SIMD JSON Parser

Phase 1: Structural Index (Week 1)

// 1. Load 64 bytes (AVX-512) or 32 bytes (AVX2)
// 2. Generate 7 structural bitmasks
// 3. Extract positions via tzcnt
// 4. Build structural index arrays

Phase 2: UTF-8 Validation (Week 1)

// Vectorized validation
// Check: overlong, surrogates, invalid codepoints
// All branchless

Phase 3: String Unescaping (Week 2)

// Find all strings via quote positions
// Vectorized unescape for common escapes
// Scalar fallback for \uXXXX

Phase 4: Number Parsing (Week 2)

// Parse integers/floats from structural index
// Use fast float parsing (lemire/fast_float)

The Counterpoint

• Complexity: 10x more code than RapidJSON
• Maintenance: SIMD intrinsics are hardware-specific
• Edge cases: UTF-8 edge cases are brutal
• Diminishing returns: Network/disk often bottleneck

Practical Takeaway

Don’t write from scratch. Use simdjson (C++), pysimdjson (Python), go-simdjson (Go), simd-json (Rust).

If you must write: Start with structural index from bitmasks. That’s 80% of the speedup. UTF-8 validation and string unescaping are the remaining 20%.

Benchmark: simdjson_benchmark --json your_file.json vs your current parser.

---

Sources: Lemire & Kostya (2019) Parsing Gigabytes of JSON per Second; simdjson GitHub; Lemire & Kaser (2021) Validating UTF-8 in Less Than One Instruction Per Byte; fast_float (lemire); RapidJSON benchmarks; Intel Intrinsics Guide; ARM NEON Programmer’s Guide.