Evolving Pinterest’s Embedding Retrieval Platform

Pinterest cut memory usage by 10x and CPU costs by 40% on a 5-billion embedding cluster simply by moving its vector index from RAM to cheap SSDs — without destroying query latency. 👇

As Pinterest’s corpus expanded toward tens of billions of embeddings, standard vector search became prohibitively expensive. Standard algorithms like HNSW require holding every vector in RAM to keep lookups fast. Scaling that model meant infrastructure costs grew linearly with every new Pin added to the platform.

To break this memory wall, Pinterest overhauled its in-house search platform, Manas. They compressed vectors with low-bit quantization, shifted indices to NVMe SSDs, and updated the query engine to handle multi-vector models.

They started by tackling raw index size. Vector search typically relies on 16-bit or 32-bit floats. Pinterest compressed these into compact integers using Scalar Quantization. To prevent the CPU from bottlenecking on decoding vectors before computing distances, they built Linear Scaling SQ. This method scales vectors purely by multiplication, allowing CPU SIMD instructions to calculate distances directly on 8-bit integers without a decoding step — cutting query compute by 10% to 15%.

Next, they moved the bulk of the index off RAM and onto NVMe SSDs. Disk reads take microseconds instead of nanoseconds, so issuing too many random reads stalls a query. Pinterest adopted SPANN, an algorithm that keeps a small centroid index in RAM while storing candidate vector lists on disk. They tweaked it by compressing the on-disk vectors with Product Quantization while keeping centroids at full precision. This change made searches 4.5x faster than basic SPANN and gave 3x the QPS of DiskANN at one-third the latency.

Finally, Pinterest moved beyond single-vector scoring. Traditional two-tower models collapse an entire Pin or query into one vector, losing token-level detail. Pinterest integrated late-interaction models like ColBERT, representing documents and queries as lists of vectors. The Manas query parser was updated to split multi-vector queries and run parallel approximate nearest neighbor searches, scoring results by finding the maximum similarity between query and document tokens.

Quantization alone cut index memory footprints by over 50% and lowered serving costs by 20% to 30%. Meanwhile, switching a 5-billion embedding production use case to SSD-based SPANN delivered 40% CPU savings with less than a 5% drop in recall.