Do your metrics tell a story?

This article was cross posted at The ITSM Review and KPI Library Expert blogs.

Do your service management metrics tell a story? No? No wonder nobody reads them.

Do your #metrics tell a story? No? No wonder nobody reads them. #ITSM #ITIL #CIO
— Dan Kane (@hazyitsm) July 30, 2013

That was a tweet I posted a few weeks ago, and it’s had some resonance. I know that during my practitioner days, I missed many opportunities to tell a compelling story. I wanted everyone else to get the message I was trying to communicate, and couldn’t figure out why my metrics weren’t being acted upon. I had a communications background before getting into IT, so I should have known better.

Facts are not the only type of data

I’ve blogged about metrics a few times before. In “Lies, Damned Lies, and Statistics: 7 Ways to Improve Reception of Your Data” I shared a story about how my metrics had gone astray. I was trying to make a point to reinforce my perspective on an important management decision. In what became a fairly heated meeting, I found myself saying at least three different times, “the data shows…” Why wasn’t it resonating? Why was I repeating the same message and expecting a different result?
Go back and read that article to see how it resolved. The short answer: I lost.

I’d love to live in a world where only objective, factual data is considered when making decisions or influencing others; but we have to recognize two important realities:

1. Other types of data, especially personal historical observations that often create biases, are more powerful than objective data ever could be.
2. Your “objective” factual data can actually reduce your credibility, if it is inconsistent with the listener’s personal observations. As the information age moves from infancy into adolescence, we are becoming less trusting of numbers, not more.

So, giving reasons to change someone’s mind is not only ineffective, it can also make things worse. Psychological research indicates that providing facts to change opinion can cement opposing opinion more deeply than before.
Information, whether accurate or not, can be found that backs up almost any perspective. Why should I trust your data any more than the data I already have? Read the comments section from almost any news story about a controversial subject. How many minds get changed?

We need a reason to care

Why should I pay attention to, act on, or react to, your metrics if there is no compelling reason for me to do so? We have to give our audience a reason to care. We want the audience of ITSM metrics to do something as a result of the metrics. The metrics should tell a story that is compelling to your intended audience.

Let’s look at a fairly common metric – changes resulting in incidents. Frequently we look at the percentage of changes that generated major incidents (or any incidents at all). Standing alone, what does this metric say? Maybe it shows a trend of the percentage going up or down over time. Even so, what action or decision should be made as a result of that data? Without context we can look for several responses:

• Service Desk Manager: “Changes are going in without proper vetting and testing.” 
• Application Development Manager: “We need to figure out why the service desk is creating so many incidents.” 
• IT Operations Director: “Who is responsible for this?” 
• CIO: “zzzzzzzz” 

Who has the appropriate response? The CIO of course (and not just because she’s the boss)! The reality is that the metric means nothing at all. Which is kind of sad really, since there may actually be something to address.

Maybe the CIO will initiate some sort of action, but not until she hears a compelling story to accompany the metric.If the metric itself doesn’t tell the story, decisions will be made based on the most compelling anecdote, whether or not it is supported by the metric.

Metrics need to tell a story

At a new job around 15 years ago, I inherited a report that had both weekly (internal IT) and monthly (business leadership) versions. Since the report was already being run, I assumed it must be useful and used. The report consisted of the standard ITSM metrics:

• number of calls opened last month vs. historical 
• incident response rate by team and priority 
• incident resolution rate by team and priority 
• highest volume of incidents by service 
• etc. 

However after a few months I realized that nobody paid attention to these reports, which surprised me. According to ITIL these are all good metrics to pull. I saw useful things in the data, and even made some adjustments to support operations as a result. However, my adjustments were limited in scope, and the improvements I saw initially didn’t hold and so everyone simply went back to the “old ways”. The Help Desk team that reported to me did experience a sustained significant improvement in their first contact resolution rate, but all other areas of support saw nothing but modest improvements over time.

The fact is that the reports didn’t tell a compelling story. There were other factors as well, but looking back now I can see that the lack of a consistently compelling metrics story held us back from achieving the transformation for which we were looking.

So your metrics need to tell a story, but how?

The traditional ITSM approach to presenting data does a poor job at changing minds or driving action, and it can actually strengthen opposing perspectives. Can you think of an example where presenting numbers drove a significant decision? Most likely, the numbers had a narrative that was compelling to the decision maker. It could be something like, “our licensing spend will decrease by 25% over the next three years, and 10% every year after.” That would be a pretty compelling story for a CFO decision maker.

In my next article, we’ll look at how metrics can tell a compelling story.