Here at Shopify, we like data. One of the many tools in our data toolbox is StatsD. We've been using StatsD in production at Shopify for many months now, consistently sending multiple events to our StatsD instance on every request. What is StatsD good for? In my experience, there are two things that StatsD really excels at. First, getting a high level overview of some custom piece of data. We use NewRelic to tell us about the performance of our apps. NewRelic provides a great overview of our performance as a whole, even down to which of our controller actions are slowest, and though it has an API for custom instrumentation I've never used it. For custom metrics we're using StatsD. We use lots of memcached, and one metric we track with StatsD is cache hits vs. cache misses on our frontend. On every request that hits a cacheable action we send an event to StatsD to record a hit or miss. Caching Baseline (Green: cache hits, Blue: cache misses) Note: The graphs in this article were generated by Graphite, the real-time graphing system that StatsD runs on top of. As an example of how this is useful, we recently added some data to a cache key that wasn't properly converted to a string, so that piece of the key was appearing to be unique far more often than it was. The net result was more cache misses than usual. Looking at our NewRelic data we could see that performance was affected, but it was difficult to see exactly where. The response time from our memcached servers was still good, the response time from the app was still good, but our number of cache misses had doubled, our number of cache hits had halved, and overall user-facing performance was down. A problem It wasn't until we looked at our StatsD graphs that we fully understood the problem. Looking at our caching trends over time we could clearly see
NayaRead shows a summary and links to the original article. The full article is hosted on shopify.engineering.