Netflix queries an 8-billion-node graph using just 16 to 24 threads per instance while keeping multi-hop latencies under 100 milliseconds. 👇
Querying a massive graph usually forces a nasty trade-off. Shallow, wide queries like finding every device an account used suffer from massive I/O fan-out. Deep, narrow queries like tracing viewing history across multiple profiles (Account → Profile → Content) accumulate network overhead with every sequential hop. If a client manages each hop individually over the network, round trips swallow the latency budget before any real processing happens.
To fix this, Netflix built an async-first execution engine on top of gRPC to evaluate entire multi-hop graph traversals within a single request.
The query journey starts by translating incoming gRPC requests into concrete execution plans. Netflix resolves all time windows and hop limits upfront so the system never over-fetches downstream data.
To fetch connections without scanning billions of rows, the storage layer uses per-node adjacency lists. When fan-out on a node is huge, storage streams adjacency lists in 100-item batches while the query engine applies filters on the fly, stopping the read as soon as query limits are met.
Traversal moves level by level in a breadth-first pattern. For a query moving from Account → Profiles → Content, the engine identifies all profiles in Level 1. It then processes Level 2 for all profiles concurrently in one parallel batch, turning dozens of potential sequential network calls into two round trips.
Because network calls dominate latency, the engine relies on fully asynchronous execution using small, dedicated thread pools. A pool of 16 to 24 threads handles thousands of concurrent requests because no thread ever blocks while waiting on I/O. Adaptive concurrency limiters monitor storage health and dynamically step down in-flight requests if timeouts spike.
Finally, stable entities like accounts, profiles, and title metadata are cached in EVCache. The engine assigns smart TTLs that align with graph retention windows, ensuring nodes nearing deletion don't pollute cache memory.
Single-hop queries execute with a P50 latency of 15 to 30ms and a P99 under 100ms across 8 billion nodes and 150 billion edges. Complex 3-hop traversals complete with a P99 between 100 and 150ms.
Selective node caching achieves a 70% to 80% hit rate, cutting storage calls by 3x to 4x on common query paths. Non-blocking async execution slashes server footprint, allowing a cluster that once needed hundreds of threads per node to run on just 16 to 24 threads.
NayaRead shows a summary and links to the original article. The full article is hosted on netflixtechblog.com.