Are you using measures to manage, or are your measures managing you?
It’s often said that what gets measured, gets managed, so perhaps you’re surprised by the statement above. Actually, in many organisations lots gets measured and very little of that is used to manage. Measurement is crucial to learning, which in turn is the basis for better performance, so surely as performance improvement specialists we’d be measurement obsessed?
Well, yes and no.
Recently Ford estimated its formal reporting procedures cost it $1.2bn (£626m) annually. And an IBM survey of marketing managers found that 89% of them didn’t understand how their measures aligned to business objectives and that on average they spent 5.4 hours per week on measurement and reporting.
I can hear all of our Public Sector clients wondering how to reduce measurement and reporting time down to 5.4 hours per week!
So, we accept that measurement can be costly, but this metric itself is a little too basic. The most costly part of measurement is having ones in place which are technically flawed, or that drive the wrong behaviours. These are the ones that really cost organisations dear.
Here are just a couple of examples.
“Market share” is a classic: how big your market share is depends on how you define the market. A senior executive of a large European packaged goods company admitted that his marketeers were a decade late in waking up to the threat of Aldi, partly because their retail share figures hadn’t included it. Aldi had been defined as being in a different market.
Technically and behaviourally flawed
A patient dying on a trolley in an NHS hospital accident and emergency department is not counted as “dying in the hospital’s care”.
So hospitals wanting to reduce their “death in care” measure can simply leave people to die on trolleys. Yes, we know that’s extreme and unlikely, but there are thousands of measures out there which could drive the wrong behaviours when the consequences aren’t quite as extreme.
So, where have some of these problems occurred? With the ground-breaking work of Kaplan and Norton on Balanced scorecards in the 1990’s and the plethora of software companies selling measurement systems, surely we should be further forward?
We typically see two types of problems when we are asked to review and help improve organisations’ measurement systems:
1. Little, or no, “balance” in measurement
It’s rare to come across organisations that have no measurement systems in place. Many of the organisations which approach us for support have far too many, most of which are used to “feed the beast” of some HQ or Whitehall function, rather than really use the measures to manage and learn.
Fewer still have a “well balanced” set of measures in place, with a tendency to concentrate on operational output volume and time measures. One of the organisations we recently worked with had improved its operational performance measurement system (and with it some elements of performance) but had missed the fact that 25% of its staff had left as it didn’t track this metric.
There are a number of characteristics that organisations display that might suggest a Balanced Scorecard approach would add value:
- A lack of appropriate measurement systems; either technically, or behaviourally, flawed
- A lack of understanding of why a more balanced set of measurements might be of use
- A need for senior managers to debate, and agree, what is really important to measure in order to track achievement of strategies
- Numerous projects and initiatives underway, all supposedly delivering results, but a lack of clarity about how their success will be measured, or if they actually do align with organisational goals
- Multiple partners and stakeholders (often with conflicting, or differing, needs and expectations) and no way of demonstrating the linkages and trade-offs
- A multitude of “mandated measurements” (e.g. required by government, or regulators, or HQ) and a view that all of these must be strategically important
But, before you go out and start developing a Balanced Scorecard, a word of warning. Getting this approach right is crucially important. We’d probably rather see an organisation with no measurement system, than one with the wrong measurement system!
A major speciality retailer measured profitability by store and rewarded store managers on this basis. In doing so, it implicitly accepted that overall profitability was a product of individual store profitability.
Result: individual stores from the same company entered into a cut-throat competition with each other, including undercutting each others’ prices, to win the same customers’ business.
A limited attempt to implement a balanced scorecard approach
We agree that understanding, and subsequent implementation, of Balanced Scorecards has developed significantly since Kaplan and Norton published their initial work in the early 1990s. We typically see three levels of maturity and understanding in the application of Scorecards:
Organisations view the Balanced Scorecard as 4 perspectives within which to select and track a few key measurements. Typically, they take the four Kaplan and Norton perspectives and develop a set of measurements for each one. These measurements may be taken from existing KPIs, or may have to be created, in order to fill the framework. This is little more than a “dashboard” of measurements, with no real understanding of any possible relationship to organisational strategy.
There is likely to be a minimal understanding of the cause and effect relationship between components on the Scorecard. The focus is on “choosing (supposedly) balanced measures”.
Not understanding cause and effect
A major organisation in Government wanted a particularly difficult client group to attend a programme to change their behaviour. The department measured the number of clients starting the programme and even set a target for the number of starts.
Perhaps unsurprisingly this drove behaviour of recommending unsuitable people to start this course and led to attrition, or drop out rates, of more than 50% in some cases. Worst case scenario would see some parts of the service recommending hundreds more people onto the programme in the knowledge that so many would drop out, which simply exacerbated the problem.
Thankfully “start” targets have now begun to disappear in favour of “completions” and “client achievements”, although major improvements are still required to the overall measurement system.
2. Strategy Mapping
Organisations adopt the Balanced Scorecard as part of their strategic planning process and it captures the essence of their existing strategy by translating it into balanced goals and measures, with associated targets and action plans. The cause and effect relationships are captured in a Strategy Map which forms the basis for senior management to discuss linkages between objectives. The focus is on developing measures related to existing strategic goals.
Strategy Maps can be used to show how strategic objectives can be linked in a cause and effect chain. The example above is a Strategy Map (simplified) from a public sector body responsible for providing funding for education.
An agreed Strategy Map is the precursor to the development of a Scorecard, and an example of a Scorecard format is shown below.
3 – Strategic Planning
In the most mature uses of the Balanced Scorecard, it is integral to strategic planning and clearly seen as a senior management responsibility for development. Strategy Maps are developed by the senior team, driven by their vision. Strategic options are considered and debated as part of the Strategy Mapping process. A small number of goals are derived from the Strategy Map and, from these, measures and targets are defined. Action plans and supporting “projects” and “initiatives” are aligned to the goals. The focus is on developing the strategy and it must therefore be led by the management team, perhaps with some facilitation support.
Level 3 is largely what Kaplan and Norton described, but is not what the early adopters necessarily understood or implemented!
And lastly, just plain stupid!!
This was from the days when bus companies were still regulated and the only measurement, or target, which existed was adherence to the timetable. A note was sent to all bus drivers stating: “The timetable measure is critical to our success. Drivers should at all times consider whether or not they can stop to pick up passengers, if the timetable is at risk”.
Ah well, at least some things have improved!