From SSH to REST: A Security-Driven Modernization of Slack’s EMR Data Pipelines
Excerpt By 2024, Slack’s data platform had accumulated 700+ SSH-based operators orchestrating critical data pipelines. We’re talking daily search indexing that processed terabytes of data, analytics jobs powering business intelligence, the whole shebang. Every single one of these jobs required direct SSH access to production AWS Elastic MapReduce (EMR) clusters. We had a massive security…
Score breakdown
- Technical depth84
- Practical value90
- Originality100
- Writing quality100
- Source reputation85
- Recency0
- External engagement0
- On-site engagement0
More like this
Pinterest Engineering56
Securing Infrastructure at Scale: Introducing Pinterest’s Resource Provisioner Pipeline (RPP)
Pinterest Engineering·unknown·6 min read
Yelp Engineering46
mrjob: Distributed Computing for Everybody
unknown·4 min read
Etsy — Code as Craft45
Enhancing Cloud Usage Forecasting, Monitoring & Optimizing
Anthony Tambasco·unknown·4 min read
Discord Engineering27
From Single-Node to Multi-GPU Clusters: How Discord Made Distributed Compute Easy for ML Engineers
unknown·1 min read
Engradar shows a summary and links to the original article. The full article is hosted on slack.engineering.