The Hook

You want to trace every open() syscall system-wide with <1% overhead. Or drop packets matching a pattern at 10M pps in the kernel. Or profile Python functions from kernel space. eBPF runs sandboxed C in the kernel — no modules, no crashes, no reboots. Here’s how it works.

The Thesis

eBPF is “JavaScript for the kernel” — safe, verifiable, programmable kernel extensions. It powers observability (bpftrace, bpftop), networking (Cilium, Katran), security (Tetragon, Falco), and profiling (py-spy, rbperf). Understanding eBPF unlocks kernel-level superpowers without kernel modules.

The Science

What Is eBPF?

Extended Berkeley Packet Filter: Virtual machine in kernel (since 3.18).

• Register-based: 10 64-bit registers (R0-R9) + stack (512 bytes)
• Verifier: Static analysis proves safety (no loops, bounded memory, no crashes)
• Maps: Key-value stores (hash, array, LRU, perf, ringbuf) for kernel↔user communication
• Helpers: Kernel function calls (bpf_get_current_pid_tgid, bpf_probe_read, etc.)

eBPF Program Types

Type	Hook Point	Use Case
XDP	NIC driver (pre-sk_buff)	DDoS mitigation, load balancing, packet filter
kprobe/kretprobe	Kernel function entry/exit	Tracing, profiling
uprobe/uretprobe	Userspace function entry/exit	App profiling (Java, Go, Python)
tracepoint	Static kernel tracepoints	Low-overhead tracing
cgroup	cgroup lifecycle	Resource limiting, isolation
socket_filter	Socket recv/send	Traffic shaping, inspection
tc (cls/act)	Traffic control (qdisc)	Traffic shaping, mirroring

The Verifier (Safety Guarantee)

Static analysis at load time:

• No unbounded loops (max 1M instructions)
• No out-of-bounds memory access
• Valid pointer arithmetic
• Valid helper calls with correct args
• Stack usage ≤ 512 bytes
• Terminates (no infinite loops)

If verifier passes: Program cannot crash kernel.

The Protocol: Write Your First eBPF Program

Minimal XDP Drop Program (C)

// xdp_drop.c
#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <bpf/bpf_helpers.h>

SEC("xdp")
int xdp_drop(struct xdp_md *ctx) {
    void *data_end = (void*)(long)ctx->data_end;
    void *data = (void*)(long)ctx->data;
    
    struct ethhdr *eth = data;
    if ((void*)(eth + 1) > data_end)
        return XDP_PASS;
    
    // Drop all IPv4 TCP port 80
    if (eth->h_proto == bpf_htons(ETH_P_IP)) {
        struct iphdr *ip = (void*)(eth + 1);
        if ((void*)(ip + 1) > data_end)
            return XDP_PASS;
        if (ip->protocol == IPPROTO_TCP) {
            struct tcphdr *tcp = (void*)ip + (ip->ihl * 4);
            if ((void*)(tcp + 1) > data_end)
                return XDP_PASS;
            if (tcp->dest == bpf_htons(80))
                return XDP_DROP;
        }
    }
    return XDP_PASS;
}

char _license[] SEC("license") = "GPL";

Compile & Load

# Compile
clang -O2 -target bpf -c xdp_drop.c -o xdp_drop.o

# Load with iproute2
sudo ip link set dev eth0 xdp obj xdp_drop.o sec xdp

# Check
sudo bpftool prog show
sudo bpftool net list

Userspace Loader (libbpf)

// loader.c
#include <bpf/libbpf.h>
#include <bpf/bpf.h>
#include "xdp_drop.skel.h"

int main() {
    struct xdp_drop_bpf *skel = xdp_drop_bpf__open_and_load();
    if (!skel) return 1;
    
    int ifindex = if_nametoindex("eth0");
    bpf_xdp_attach(ifindex, bpf_program__fd(skel->progs.xdp_drop), 0, NULL);
    
    // Keep running
    while (1) sleep(1);
    
    bpf_xdp_detach(ifindex, 0, NULL);
    xdp_drop_bpf__destroy(skel);
    return 0;
}

Build with Makefile

# Makefile
BPF_HEADERS := /usr/include/bpf
CLANG := clang
LLC := llc

%.o: %.c
	$(CLANG) -O2 -target bpf -I$(BPF_HEADERS) -c $< -o $@

%.skel.h: %.o
	bpftool gen skeleton $< > $@

clean:
	rm -f *.o *.skel.h

Python/BCC Example (Simpler)

# trace_open.py
from bcc import BPF

bpf_text = """
#include <uapi/linux/ptrace.h>
#include <linux/sched.h>

struct data_t {
    u32 pid;
    u64 ts;
    char comm[TASK_COMM_LEN];
    char fname[256];
};

BPF_PERF_OUTPUT(events);

int trace_open(struct pt_regs *ctx, const char __user *filename) {
    struct data_t data = {};
    data.pid = bpf_get_current_pid_tgid() >> 32;
    data.ts = bpf_ktime_get_ns();
    bpf_get_current_comm(&data.comm, sizeof(data.comm));
    bpf_probe_read_user_str(&data.fname, sizeof(data.fname), filename);
    events.perf_submit(ctx, &data, sizeof(data));
    return 0;
}
"""

b = BPF(text=bpf_text)
b.attach_kprobe(event="do_sys_openat2", fn_name="trace_open")

def print_event(cpu, data, size):
    event = b["events"].event(data)
    print(f"{event.ts/1e9:.3f} {event.comm.decode()} [{event.pid}] {event.fname.decode()}")

b["events"].open_perf_buffer(print_event)
while True:
    b.perf_buffer_poll()

The Protocol: Build Observability Stack

Level 1: bpftrace (High-Level)

# Trace all opens
bpftrace -e 'tracepoint:syscalls:sys_enter_openat { printf("%s %s\n", comm, str(args->filename)); }'

# Profile on-CPU time
bpftrace -e 'profile:hz:99 { @[stack] = count(); }'

# TCP connect latency
bpftrace -e 'kprobe:tcp_connect { @start[pid] = nsecs; } kretprobe:tcp_connect /@start[pid]/ { @lat = hist(nsecs - @start[pid]); delete(@start[pid]); }'

Level 2: libbpf + CO-RE (Compile Once, Run Everywhere)

# Install
sudo apt install linux-headers-$(uname -r) clang llvm libbpf-dev

# CO-RE: Compile once, run on any kernel with BTF
clang -g -O2 -target bpf -D__TARGET_ARCH_x86 -I/usr/include/bpf -c prog.c -o prog.o
bpftool gen skeleton prog.o > prog.skel.h

Level 3: Production Patterns

Pattern	Use Case
Ring buffer	High-volume events (perf_event_output)
Ring buffer (multi-buffer)	Per-CPU buffers, zero-copy
Hash map	Key-value state (conn tracking)
LRU hash map	Auto-eviction
Array map	Fixed-size, fast lookup
Stack trace map	StackID → stack frames

The Counterpoint

• Kernel version: CO-RE needs BTF (kernel ≥ 5.2)
• Verifier frustration: Complex logic often rejected
• Kernel headers: Need matching headers or BTF
• Debugging: Limited (bpftool, bpftrace, printk)

Practical Takeaway

Start with bpftrace — one-liners for immediate value. Then libbpf + CO-RE for production tools. Use cases:

• opensnoop — file opens
• tcplife — TCP connection lifespan
• biolatency — block I/O latency
• profile — CPU profiling
• tcpretrans — TCP retransmissions

Install: sudo apt install bpftrace bcc libbpf-dev or yum install bpftrace bcc-tools

---

Sources: Brendan Gregg “BPF Performance Tools”; Cilium eBPF docs; Facebook “BPF at Scale”; Netflix “BPF at Netflix”; Cilium “BPF and XDP Reference Guide”; Linux kernel docs “BPF Documentation”; Brendan Gregg’s BPF tools repo.