Measuring stuff is hard, even when it stands still. When things change over time the situation gets even worse. I'm Ben Doyle, research scientist and data prophet at Shopify. Over the next little while I'm planning to go into some detail about how we're calculating some common ecommerce metrics (like customer count, churn rate, lifetime value, etc.) here at Shopify. Most of these metrics come down to fancy ways of counting, but the devil is in the details. For Instance... You have a table of users and a boolean flag that says whether each of them is a paying customer. Counting your customers is as simple as counting the rows where the flag is set. Or is it? What if you want to plot your growth over time? You can replace the flag with a date range and count how many of these ranges include a date in question. What if your customer definition changes, perhaps due to a change in your business model? You have to go back through your whole history, which might be a problem if your new definition relies on information you have just started collecting. If you are doing exploratory work you might not even know what data you need yet. These problems have solutions but clearly there's work to do. We need to untangle our methods and sort out our definitions. I thought I'd kick things off by clarifying the elements most basic to counting: intervals and points. Intervals and Points In the diagram above the dark blue lines represent intervals and the light blue circles represent points (the points should really only be a single pixel but are large so we can see them). In general you need intervals to measure points and you need points to measure intervals. So in a) three of the four intervals overlap with the point and in b) three of the four points overlap with the interval. States And Events In more familiar terms the space we're
NayaRead shows a summary and links to the original article. The full article is hosted on shopify.engineering.