All Measures are Proxies

I've been a stat-head my whole life. Whether it's sports stats, how much milk our cows were making, box office data, or incident data, I'm interested. So I'd like to focus this post on what data can tell us and what is cannot.

Most people are familiar with the notion of proxies - a measure that is used when you can't measure something directly. In theory, anything can be used as a proxy for anything else. For instance, the number of mirrors I have broken or black cats that have crossed my path could be a proxy for how unlucky I will be in the future.

In practice, we tend to only use a proxy when we perceive a cause and effect relationship. For instance, we use a customer survey score as a proxy for customer satisfaction. A good proxy meets the following conditions:

• Credibility - stakeholders believe that the proxy is a reasonable stand-in
• Correlation - movement of the proxy appears to mirror the movement of the item to which it is linked

In a sense, even things that we can easily measure serve as proxies for something else we can't measure. For instance, batting average in baseball is a proxy for whether a player is a good hitter. The amount of money in our bank account is a proxy for how wealthy (or poor!) we are. And so on.

Where this becomes important in ITSM is when no one agrees on how to measure something. For instance, think about the warranty measures - availability, capacity, continuity and security. There is no one agreed definition for any of these terms that would allow you to take a set of data and calculate its measure. This causes a lot of headaches as people debate what the "right" formula is to calculate availability, for instance.

The most useful way to think about these terms is to understand why we want to measure these things. Most groups want to measure these things so they can understand how well a service is performing. It also helps them communicate with the customers of their service. 

Do we need the "correct" calculation in order to do these things? No! We just need to calculate it in a way that is credible to stakeholders and that correlates with delivery of good service. As long as everyone views the calculation as reasonably approximating reality (as they see it), and helps them focus on what areas to improve, the measure is as good as it needs to be!

So the next time you're struggling with metrics, ask yourself why you're trying to measure something. As Lean principles state, overproduction is waste. Don't try to produce a perfect measure when a decent one will let you do what is needed to deliver value!