menu SIGN IN
Stacey Barr
Stacey Barr
Performance Measure Specialist

Stacey Barr Pty Ltd
Websites:

3 Good Reasons To Avoid Indexes And Scores

Posted about 7 years ago

You might think that rolling up your performance measures or KPIs into weighted indexes or scores is a great strategy to simplify your dashboard, to deal with lots of measures, or to create proxies for hard-to-measure goals. Think again.

An index is the term we give to a measure that is computed by taking a weighted sum or weighted average of a collection of other measures.

For example, you might take a dozen or so different measures to do with productivity, and create a weighted average of those measures to give you an Overall Productivity Score. Or you might create a Business Unit Performance Index for each of the business units in your company or organisation.

It might sound like a good idea, but it isn’t. Here are three reasons why indexes and scores are a bad idea:

Reason #1 is that you lose focus. If you’re using an index because you have too many measures to digest in one sitting, then the problem is that you have too many measures! Performance measurement is not about measuring everything you can, it’s about measuring only those things you can and should and will do something about.

Suggestion #1: Test the value of each of your performance measures using this checklist of criteria of excellent performance measures.

Reason #2 is that you lose the real signals. The signal your index gives you will be a diluted average of the signals of all the underlying measures. If you have one underlying measure tracking very well and another underlying measure tracking very poorly, they will average out in the index.

Suggestion #2: Rather than summarising every measure into an index or score, use simple formating to highlight the signals in each underlying measure, to make it easy to focus just on those measures that need attention. Learn how from dashboard guru, Stephen Few, at www.perceptualedge.com.

Reason #3 is that you lose sight of the tangible results you are trying to monitor. When you look at an index or score, it’s usually a rolled-up summary of a broad performance area, like productivity or customer service or financial performance. But these broad performance areas aren’t tangible or actionable. It’s the specific performance results that lie within these domains of performance that you need to always keep your eye on. They are the reason you’re measuring anything at all.

Suggestion #3: Align your performance measures and your strategic initiatives to specific performance result statements. That way you maintain the actionable link between strategic targets and strategy execution using the feedback from the relevant measures. Learn how to do this with the PuMP Performance Measure Blueprint.

TAKE ACTION: Rather than just taking my word for it, look at one or two of the indexes or scores you currently use, and determine what action you might take based on what it’s telling you. Now, look at the signals in the underlying measures and see what action you would take based on what they are telling you. Is the action the same? If not, which action seems like the right action to take?

Comments (6)

Greg Glynn
Greg Glynn
NSW Businesslink

Well done Stacey – It makes good sense. I guess we need to also think about limiting the things we measure too.

Posted about 7 years ago | permalink
Karel van der Poel
Karel van der Poel
Founder Mirror42

Great article Stacey. In reply to Greg Glynn: you should limit number of KEY indicators, not the number of metrics you measure.

Example: Retaining customers is an important metric to optimize before you start spending marketing dollars to acquiring new customers. But is does not mean that you stop measuring retention once you have optimized it.

My advice is to always measure your metrics, track historic trends and maintain your performance history. Just dont call everything “KEY”. Your “KEY” indicators are a “select few” related to your “CURRENT GOALS”. Once you hit your goals, your “KEY” indicators change.

Posted about 7 years ago | permalink
joyce maphorisa
joyce maphorisa
consultant at PGC

Great idea espacially that its not easy to drill down to the area that needs actions/decisions if everything is bundled up in an index. How does one measure something like the totality of ratings on attributes that lead to customner satisfcation without being tempted to make it an index? and without a list of things to measur- what about a rating on a customer satisfaction survey?

Posted about 7 years ago | permalink
joyce maphorisa
joyce maphorisa
consultant at PGC

Great idea espacially that its not easy to drill down to the area that needs actions/decisions if everything is bundled up in an index. How does one measure something like the totality of ratings on attributes that lead to customner satisfcation without being tempted to make it an index? and without a list of things to measur- what about a rating on a customer satisfaction survey?

Posted about 7 years ago | permalink
Stacey Barr
Stacey Barr
Performance Measure Specialist

Greg – I absolutely agree with Karel’s advice about KPIs. We shouldn’t forget that KPIs are just one type of performance measure, and not the only type. They are the “key” measures.

I would still suggest that we limit how many measures we have “on our radar”, in the sense that too much information, and trying to improve too many things at once, can be far worse than focusing on too few.

Here’s an article discussing how many KPIs you should have:

http://kpilibrary.com/topics/how-many-kpis-do-you-need

Posted about 7 years ago | permalink
Stacey Barr
Stacey Barr
Performance Measure Specialist

Joyce, I usually handle measurement of customer satisfaction this way:

1) include a question asking for the customer’s OVERALL satisfaction with the service you provide

2) include up to a dozen additional questions, asking for satisfaction with specific attributes of service e.g. timeliness, accuracy, professionalism (but make sure you derive these from what the customer says is important)

3) With a bit of analysis you can see which attributes of service mostly drive overall satisfaction.

No need for bundling things into an index! Here’s an article with more info about useful customer surveys:

http://kpilibrary.com/topics/5-steps-to-simple-and-actionable-customer-surveys

Posted about 7 years ago | permalink

Log in to post comments.