Most real-life goals can't be easily expressed with few "easily trackable" metrics. Take example from the article, customer satisfaction. That's half of the problem; the other half is that metrics can be gamed, and they'll end up being gamed accidentally or for profit if you're not careful. Like, the easiest way to make a system stable is to make it so painful to use that people don't use it - and if they don't use it, they can't break it. Or, the example of return tracking from the TFA.
I read a lot about how data-driven companies measure this and that, often through questionable, privacy-violating measures. What I don't read about is how do these companies ensure the metrics are actually valid - that they're correctly sampling the population[0], or that they're measuring what the authors think they're measuring, or that they're not being misreported or otherwise gamed (very common if the value of a metric impacts someone's career or even workload).
--
[0] - e.g. voluntary surveys usually don't, telemetry increasingly doesn't either as more and more people are aware of it and disable it.
Don't choose things that can't be tracked quantitatively. Track things quantitatively and then make qualitative decisions.
At Square, we had "Make Commerce Easy". This was qualitative and thus up for interpretation. It became even vaguer when it became "Economic empowerment".
These things can't and shouldn't be tracked as if they are quantitative metrics but instead used as guiding stars.
So the trick is just to separate the two and as I said. Use data metrics for optimizing and not for creating or measuring of qualitative goals.
> Track things quantitatively and then make qualitative decisions
Decisions are often inherently quantitative, e.g., you allocate an actual dollar amount to a budget. Or a project go/no-go is a crisp binary decision. I think you mean don't make decisions directly based on quantitative metrics but on qualitative assessments informed by quantitative metrics.
I don't distinguish that hard the second I get into the qualitative so many factors (like experience) inform the quantitative work.
Sometimes the qualitative goals is created because of the quantitative data.
What you are saying is certainly also what I mean but I don't think I only mean that but I see what you are saying and yeah it might just be my english.
The problem is when companies choose qualitative and thus interpretational metrics.
Just stay to the simple easily trackable and quantitative ones and the make it your job to interpret them qualitatively.