Accuracy vs. precision: what it means for your product decisions
What’s the difference between accuracy and precision? My Dad asked me this at the age of 10. I told him it was a stupid question. However, it’s increasingly important as we become inundated with data. There is a risk of PMs / orgs / companies over-utilizing a “data-driven” approach to the point where decision makers neglect pursuing step-function changing ideas because the “data doesn’t support it.” A healthy use of this data requires a keen understanding of when to ignore it. This is where precision vs. accuracy enters the picture.
Wikipedia nicely sums up the technical difference (I added some formatting for emphasis):
Accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity’s actual (true) value. The precision of a measurement system, also called reproducibility or repeatability, is the degree to which repeated measurements under unchanged conditions show the same results.
Here is a visual depiction:
To summarize, the more precise you are, the tighter the bell curve. The more accurate you are, the closer the center of your bell curve is to the actual true reference value. This is highly relevant for product stakeholders because we are confronted daily with making tradeoffs between:
a) Building features we have strong conviction in based off of past success and internal data (eg, we can precisely estimate impact), and
b) Taking bigger risks on opportunities that are more difficult to quantify, but that we believe will drive step function changes in the business (eg, accurately nail big opportunities).
Why does it matter?
- Precision is seductive.
In the age of big data, it’s an increasingly effective tool of the Sirens. It’s easy to be seduced into building features whose impact you’ll be able to predict precisely. For example, in the gold rush days of FB platform apps, many developers spent significant resources optimizing FB request flows. Why? Because it was alluringly simple to precisely predict the impact tweaks would have, execute them and measure the results. No doubt, the human brain gets an inordinate satisfaction out of this - it’s like magic! You said X would happen, and you were right (X*.98 happened)! It might be fun to be a prognosticator, but …
- Precision has an opportunity cost.
There is a significant opportunity cost in consistently prioritizing precision over accuracy. Accuracy is about launching what the market needs, precision is about optimizing and delivering relentlessly on it. Unless you’ve nailed the former, material effort on the latter is going to be wasted because you’re optimizing something too far from the true north (the accurate goal) you should be pursuing. As Andrew Chen pointed out, startups that don’t have product/market fit don’t need growth hackers - that’s akin to focusing on precision when you haven’t nailed accuracy yet.
To sum it up, when you’re building product, it’s important to be cognizant of whether you’re striving for accuracy or precision (and whether that is appropriate within the context of your product / org / company). If you find yourself carefully justifying feature decisions with overly tidy, buttoned-up, high confidence items, ask yourself whether you’re sacrificing accuracy for precision. It’s better to be directionally accurate than precisely wrong.