How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache

Cloudflare freed 100 terabytes of RAM across its fleet by trimming down its DNS cache—and somehow made query lookups 19% faster in the process. 👇

Big Pineapple, Cloudflare’s DNS platform behind 1.1.1.1, stores over 250 billion cache entries at any given moment. At that scale, wasting just one byte per entry discards 250 gigabytes of memory across the fleet. Standard Rust data structures, memory alignment padding, and heap fragmentation were silently consuming gigabytes of unnecessary space.

To fix this, Cloudflare overhauled how cache entries are laid out in memory, using five targeted Rust optimizations to strip away overhead without sacrificing lookup speed.

Replace vectors with boxed slices: A Rust `Vec` and `String` store a pointer, a length, and a capacity field to allow future growth. Because cached DNS entries never change once written, that extra 8-byte capacity field and its pre-allocated heap buffer are entirely wasted. Cloudflare swapped them for `Box<[T]>` and `Box<str>`. Eliminating 8 vector fields per entry saved 64 bytes per entry and over 15 terabytes fleet-wide.

Consolidate section lists: DNS responses have answer, authority, and additional sections. Instead of storing three separate heap-allocated lists—each requiring an 8-byte pointer and 8-byte length—they merged them into one list referenced by 2-byte (`u16`) offsets. This saved 28 bytes per entry and removed extra Rust struct padding.

Infer identical record owners: Most DNS records belong to the domain being queried. Cloudflare stopped storing the owner name on records when it matched the cache key, inferring it on the fly during lookups instead. Only distinct domain names (like CNAME targets) get an explicit heap pointer.

Flatten enum padding: Rust enums take the size of their largest variant. Big Pineapple's record enum was 144 bytes to fit rare `NAPTR` records, forcing common 4-byte `A` records to waste 120 bytes on padding. Boxing large variants helped, but created separate heap allocations that fragmented memory across jemalloc bins.

Pack records as raw wire bytes: To solve the fragmentation, Cloudflare dropped the record enum entirely. They now serialize all records into a single contiguous byte array (`Box<[u8]>`), prefixed with 2-byte lengths. To avoid allocation churn, records are serialized into a reusable scratchbuffer first, then copied once into the final box.

Storing contiguous wire bytes dramatically improved CPU cache locality. It also allowed common records like `A` and `AAAA` to be copied straight to outgoing network buffers without re-parsing.

In benchmarks, the per-entry memory footprint dropped 56% from 953 bytes down to 420 bytes. In production, p99 memory usage per server dropped from 9.3 GB to 5.3 GB. Across the entire fleet, Cloudflare freed 100 terabytes of RAM, boosted cache insert throughput by 43%, and cut lookup latency by 19%.