Your web application is likely rendering requests when the requesting client has already disconnected. Eric Wong helped us devise a patch for the Unicorn webserver that will test the client connection before calling the application, effectively dropping disconnected requests before wasting app server rendering time. The Flash Sale A common traffic pattern we see at Shopify is the flash sale, where a product will be discounted heavily or only available for a very short period of time. Our customer's flash sales can cause traffic spikes an order of magnitude above our typical traffic rate. This blog post highlights one of the problems dealing with these traffic surges that we solved during our preparation for the holiday shopping season. In a flash sale scenario, with our app servers under high load, response time grows. As our response time increases, customers attempting to buy items will hit refresh in frustration. This was causing a snowball effect that would contribute to reduced availability. Connection Queues Each of our application servers run Nginx in front of many Unicorn workers running our Rails application. When Nginx receives a request, it opens a queued connection on the shared socket that is used to communicate with Unicorn. The Unicorn workers work off requests in the order they're placed on the socket’s connection backlog. The worker process looks something like: The second step takes the bulk majority of time of processing a request. Under load, the queue of pending requests sitting on the UNIX socket from Nginx grows until it reaches maximum capacity (SOMAXCONN). When the queue reaches capacity, Nginx will immediately return a 502 to incoming requests as it has nowhere to queue the connection. Pending Requests While the app worker is busy rendering a request, the pending requests in the socket backlog represent users waiting for a result. If a users hits refresh, their browser closes the current connection and their new connection enters the end of the queue (or nginx returns a
NayaRead shows a summary and links to the original article. The full article is hosted on shopify.engineering.