A couple of weeks ago, I posted a two part series detailing how we do data QA at Semantics3. In the days that followed, I’ve had people get in touch to discuss the best ways to establish or improve data/product quality initiatives at their own organizations. For some, the impetus was a desire to stem customer churn, and for others just to make customers happier.
Here were some of the common discussion points from these conversations:
- Should we hire a dedicated QA analyst to solve our data quality issues?
- What sort of profile should such a hire have?
- How do you decide what your data QA budget should be?
- How do you draw a line between automation and manual effort?
- We don’t want to build on heuristics because they don’t scale – any alternatives?
The inherent theme in all of these questions was that everyone looking for a framework to think about quality analysis. So, while my previous posts addressed specific ways in which we do data QA at Semantics3, in this article, I’d like to speak to the philosophy behind them.