Backend

Up: Portable Microservices Ready for the Cloud

Rebuilding Netflix Video Processing Pipeline with Microservices

This is the first blog in a multi-part series on how Netflix rebuilt its video processing pipeline with microservices, so we can maintain…

To infinity and beyond: enabling the future of GitHub's REST API with API versioning

We’re introducing calendar-based versioning for our REST API, so we can keep evolving our API, whilst still giving integrators a smooth migration path and plenty of time to update their integrations.

Introducing Domain-Oriented Microservice Architecture

Seamlessly Swapping the API backend of the Netflix Android app

How we migrated our Android endpoints out of a monolith into a new microservice

Refactoring Backend Engineering Hiring at Slack

For anyone who’s ever been involved in the hiring process, it’s no easy feat — particularly in a growing company. To get hiring practices right, it takes iteration based on feedback — both on the internal processes within your company as well as on the external process a candidate experiences. Continuously improving hiring is important for a host of…

Using GraphQL for High-Performing Mobile Applications - Shopify

GraphQL is the syntax that describes data that a client asks from a server. The client, in this case, is a mobile application. GraphQL is usually compared with REST API, a common syntax that most mobile application developers use. We will share how GraphQL can solve some of the pain points of REST API in mobile application development and discuss tips and best practices that we learned at Shopify by using GraphQL in our mobile applications. Why Use GraphQL? A mobile application generally has four basic layers in the codebase: Network layer: defines the connection and the server to connect to send/receive data. Data model layer: translates data coming from the network layer to understandable data for local app models. View models layer: translates data models to understandable models for the user interface. User interface layer: presents/receives data to/from the user. A network layer and data model layer are needed for an app to talk to a server and translate that information to view layers. GraphQL can fit into these two layers and base a data graph layer and solve most of the pain points mobile developers used to have when using REST APIs. One of the pain points when using REST APIs is that the data coming from the server should be mapped many times to different object types in order to be presented on the screen or vice versa from input in the screen to be sent to the server. Simpler apps might have fewer of these mappings depending on if the app has a local database to store data or if the app is online only. But mobile apps surely have the mapping to convert the JSON data coming from an API to a class object (for example, Swift objects ). When working with REST endpoints these mappings are basically matching statically typed code with the unstructured JSON responses. In other words, mobile developers are asked to hard code the type of a field and

How and Why Netflix Built a Real-Time Distributed Graph: Part 3 — Querying the graph with gRPC…

In praise of “boring” technology | Spotify Engineering

In this article I will explain how Spotify uses different mature and proven technologies in our backend service eco-system and architecture, and why we do so. In addition, this article will also attempt to explain when Spotify has chosen not to use certain proven technologies, the reasons why and the associated pitfalls associated with each.

Scalable User Privacy | Spotify Engineering

At Spotify, we have a complex and diverse data processing ecosystem. Our backend infrastructure handles millions of requests per second, which are processed by over a thousand (micro)services. Our batch pipeline environment is equally complex and diverse; we run thousands of jobs written in a variety of frameworks such as , ,  and . The data processed by these services and pipelines power critical parts of our organization. We use data to calculate royalties, run A/B tests, process payments, serve playlists and suggest new tracks to our users.

Improving Critical Infrastructure Rollouts | Spotify Engineering

Spotify began using Docker with a few prototype services in 2014. We upgraded and configured it many times since and have almost every time come across issues that were often hard to detect and fix. When the number of backend services running on Docker was low, the impact of these issues was small. As Docker adoption grew so did the risk and impact of faulty Docker changes until they reached unacceptable levels. In October 2016, we deployed a bad configuration change that significantly affected the user experience.

Navigating Recurring Payments in India: A Backend Perspective - Shopify

To address the new RBI framework for recurring payments in Shopify’s billing platform, we worked with a local payment provider that could accommodate both card-based payments as well as a local payment method called Unified Payments Interface (UPI).

Testing of Microservices | Spotify Engineering

Most people would say that we write tests to verify that things work as we expect them to.

BRZ

The Austrian Business Service Portal (USP), developed and operated by the Federal Computing Center (BRZ), is the central eGovernment platform connecting businesses to over 130 digital public services. To securely manage access for more than 2 million users, USP now relies on a modern architecture powered by Keycloak, microservices, and GitOps.

Lunar

Lunar, a Scandinavian online bank that embraced cloud native tech early on, has been running Kubernetes since 2017. Today, they have over 250 microservices distributed across three clouds.

Backend infrastructure at Spotify | Spotify Engineering

backend infrastructure at Spotify. Our backend infrastructure is very much work in progress – in some areas we have come a long way and in others we have just started.

Achieving fault-tolerance with intelligent daemons | Spotify Engineering

Powering the Spotify service is a backend of dozens of different, specialized service implementations. For example, we have a playlist system that is responsible for the storing and retrieval of users playlists, a user service responsible for login, and we have an ad system with advertisements and the surrounding business logic.

Painting a Picture of Your Infrastructure in Minutes | Spotify Engineering

As a Security responder, it is important to have access to relevant information quickly in order to get a more complete picture of an ongoing or potential incident. Modern microservice architecture usually means that the number of services, and thus hosts and related assets, are much higher than they used to. This means that getting an overview of them is no longer a trivial task and as resources become more and more ephemeral, the inventory is not only large in volume but it’s also rapidly changing.

Large-Scale Automated Dependency Analysis Across Uber's Service Mesh

Behind The Code: Jennice Colaco, Backend Developer - Shopify

Behind the Code is an initiative with the purpose of sharing the various career stories of our engineers at Shopify, to show that the path to development is non-linear and quite interesting. The features will showcase people just starting their careers, those who made career switches, and those who've been in the industry for many years. Enjoy!

CVTE

CVTE, an electronics company that embraced cloud native tech early on, has been running services on Kubernetes for years. Today, they have a large number of microservices distributed across regions.

Techbytes: Restoring Confidence in Microservices – Tracing That’s More Than Traces | Spotify Engineering

Ben Sigelman, CEO of LightStep and co-author of the OpenTracing standard, stopped by our New York office to talk distributed tracing – the art and science of making distributed traces valuable – and explain how it can redefine our relationship with microservices.