Topic
20 articles
Cloudflare Research is building a global consensus service called Meerkat that uses a new consensus algorithm called QuePaxa. We plan to use Meerkat to build a strongly consistent, fault-tolerant key-value store, and other applications.
Our hardware infrastructure comprises millions of machines, all of which generate logs that we need to process, store, and serve. The total size of these logs is several petabytes every hour. The o…
mrjob: Distributed Computing for Everybody Dave M Search and Data-Mining Engineer Oct 29, 2010 Ever wonder how we power the People Who Viewed this Also Viewed… feature? How does Yelp...
Monitoring Cassandra at Scale Joseph Lynch, Software Engineer Jun 1, 2016 At Yelp we leverage Cassandra to fulfill a diverse workload that seems to combine every consistency and availability tradeoff...
AI networks play an important role in interconnecting tens of thousands of GPUs together, forming the foundational infrastructure for training, enabling large models with hundreds of billions of pa…
Introducing PaaSTA: An Open, Distributed, Platform as a Service Kyle Anderson, Site Reliability Engineer Nov 10, 2015 As an Operations engineer, my first priority is to keep the site up....
Within large-scale services, durable storage, distributed leases, and coordination primitives such as distributed locks, semaphores, and events should be strongly consistent. At Meta, we have histo…
Facebook Ordered Queueing Service (FOQS) is a fully managed, distributed priority queueing service used for reliable message delivery among many services. FOQS has evolved from a regional deploymen…
We’re rolling out MySQL Raft with the aim to eventually replace our current MySQL semisynchronous databases. The biggest win of MySQL Raft was simplification of the operation and making MyS…
Over the years, Meta has invested in a number of storage service offerings that cater to different use cases and workload characteristics. Along the way, we’ve aimed to reduce and converge th…
At Shopify, we’ve long used database replication for redundancy and failure recovery, but only fairly recently started to explore the potential of replicas as an alternative read-only data source for applications.
Our Sidekick team used LLM judges to automatically curate training data and teach domain-specific models how to refuse impossible queries.
How Instacart uses distributed Machine Learning to efficiently train thousands of models in production Author: Han Li At Instacart, we take pride in offering a diverse range of machine learning (ML) products that empower every...
By Michael Prescott This blog post is the last segment in a three-part series that outlines our innovative approach to overcoming inventory challenges, delving into the application of product design, machine learning, and engineering technology....
Learn advanced database reconciliation patterns for Stripe integration. Explore queue-based architectures, event-driven sync, and robust error handling to maintain product data consistency at scale.
5 minute read Over the course of Shopify's 13-year codebase history, the core platform has never been rewritten. That meant a slew of outdated code styles, piling atop of one another and without a lot of consistency. By 2012, our CEO Tobi created a draft Ruby style guide to keep up with the growth. Unfortunately, it never became embedded in our programming culture and many people didn't even know it existed.
A failed payment request means a Stripe user did all the work to get a customer to pay, but lost the sale because of us. We never want this to happen. To minimize failed payments for our users, we target 99.9995% reliability. Maintaining that level of reliability while processing around 1.6% of global GDP is an exceptionally difficult technical challenge, but most of us who join Stripe’s infrastructure teams are here because we love technical challenges.