At Spotify, we have a complex and diverse data processing ecosystem. Our backend infrastructure handles millions of requests per second, which are processed by over a thousand (micro)services. Our batch pipeline environment is equally complex and diverse; we run thousands of jobs written in a variety of frameworks such as , , and . The data processed by these services and pipelines power critical parts of our organization. We use data to calculate royalties, run A/B tests, process payments, serve playlists and suggest new tracks to our users.
NayaRead shows a summary and links to the original article. The full article is hosted on engineering.atspotify.com.