Lessons Learned From Running Apache Airflow at Scale - Shopify
Shopify shares some of the lessons we learned and solutions we built in order to run Airflow at scale.
Discussion around the web
- Hacker News285 · 182 💬
Score breakdown
- Technical depth81
- Practical value100
- Originality85
- Writing quality100
- Source reputation86
- Recency0
- External engagement36
- On-site engagement0
More like this
Spotify Engineering49
How Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020 | Spotify Engineering
Neville Li, Claire McGinty, Sahith Nallapareddy, & Joel Östlund·unknown·11 min read
Shopify Engineering54
Shopify Invests in Research for Ruby at Scale - Shopify
unknown·6 min read
Shopify Engineering51
How to Reliably Scale Your Data Platform for High Volumes - Shopify
unknown·12 min read
Shopify Engineering64
SkyPilot at Shopify: Multi-cloud GPUs without the pain (2026) - Shopify
unknown·8 min read
Engradar shows a summary and links to the original article. The full article is hosted on shopify.engineering.