Every company, no matter if it has just started on their CI journey or has a mature and strong improvement culture, wants to measure the effectiveness of their CI programme. In other words, companies want to know: What is going well? What can be done better to identify improvement opportunities in their CI programme? What is their return on investment on CI activities? How can they extract more value from their CI initiatives?
Companies use a variety of performance measures such as hard financial benefits, non-financial benefits, number of improvement projects, rapid improvement/kaizen events, problem-solving sessions completed, and CI training sessions delivered etc. They also track and set the targets for building their CI infrastructure in terms of the number of CI practitioners, Green Belts (GBs), and Black Belts (BBs). In addition to these quantitative metrics, many companies use employee/customer surveys and CI assessment tools to measure the success and maturity of their CI practices.
However, as Albert Einstein once said, “Not everything that can be counted counts, and not everything that counts can be counted”. While we all agree that “what gets measured gets managed”, it is also true that too much focus on one or two performance measures drive wrong behaviours. As Eli Goldratt said, “Tell me how you measure me, and I will tell you how I will behave”. Let’s look at the following two examples:
Hard financial benefits: Too much focus on financial benefits, particularly at the start of CI journey from the senior management, can derail the entire CI programme. It impels the CI teams to spend too much time on quantifying the benefits rather than actually doing CI. It also sends a wrong signal – the company seems to be more interested in cost savings rather than building a truly customer-focussed problem solving culture. In some situations, it is also seen that the projects with short term financial benefits are prioritised over the other projects with long term sustainable benefits. Continuous improvement is not just a one off cost-cutting exercise rather it is a multi-year journey aimed to bring about the cultural change by embedding right mindset and behaviours in the organisation.
CI Assessment Tools: Many companies use qualitative assessment tools to measure the maturity of their CI practices. Typically, these assessments score businesses or a part of the business on several CI criteria such as problem solving culture, understanding of customer requirements, managerial support, closure rate of CI projects, number of certified Green and Black Belts etc. Again, if the companies are not careful, these tools can quickly become a tick-box exercise. I have seen that many teams “managing” their maturity score really well but in reality there has been hardly any tangible improvement experienced by the customers and/or the business.
In the end, all metrics have faults and if not carefully selected, and thoughtfully applied, they do more harm than good to CI initiatives. Therefore, the fundamental questions for the companies are: what they should measure and how they should set about developing the metrics. The following 4-step approach will help you in developing robust performance measures for your CI programme:
1. Align CI measures to business objectives: Any CI programme must play a crucial role in delivering company’s short term and long term objectives. The measures of CI programme should reflect how effective CI activities have been in meeting organisation’s objectives. For example, if a company has identified customer responsiveness, on time delivery and improving quality of its products as key success factors in the market, the performance measures of CI activities at business, department and team levels should mirror these priorities. Without this clear link, companies end up trivialising their CI programmes and risk losing the support of senior management and the front line people. In one of the CI deployments, I have seen how this missing link led to the CI programme becoming mere a low value 5S (workplace organisation) programme with no tangible benefits to the business and its employees.
2. Establish a balanced set of CI measures: Having a balanced scorecard for CI programme helps the company to achieve a trade-off between conflicting priorities of customer satisfaction, cost, quality and delivery. CI activities must be aligned to improve performance of the metrics defined within each of these categories. If we take an example of on time delivery (OTD), some of the CI measures include:
- Number of DMAIC/A3s/Just Do It activities completed linked to OTD metric
- % improvement in OTD performance through CI activities
Similarly, we can define the CI measures for Customer, Quality and Cost. In addition to measuring the impact of CI programme on business objectives, it is also essential to measure the organisational CI capability in terms of the number of GBs/BBs, number of newly certified belts, number of people trained on CI, CI practices maturity etc. The scorecard should have a good mix of the CI impact and capability measures.
A balanced scorecard gives us deeper insights into what is going well and where the major gaps are in the CI programme. For example, if a business has strong CI capability but is not able to move the needle on key business metrics, the scorecard can help you in asking very pertinent questions such as: why are we not seeing the desired benefits? Are we prioritising right CI projects? Are we closing the projects on time? If not, why do we have low project closure rates? Are the right CI resources working on the right projects?
3. Set SMART targets for the CI measures: One of the key factors for setting the targets is at what stage your CI journey is in. If a company has just launched their CI programme, the focus should be on how quickly to build the CI foundation and the required infrastructure. Therefore, in early stages the emphasis should be more on the CI capability measures rather than on the impact measures. As the programme matures, the focus can gradually be moved to measuring the impact of CI initiatives on business performance. As CI becomes business as usual, additional metrics such as the number of active GBs/BBs (e.g. Certified BBs/GBs who have completed at least one improvement project in the past one year), number of projects successfully replicated across the sites/businesses etc. can be introduced to maximise the value from CI programmes.
4. Regularly assess and refine the CI measures: Setting up CI measures is not a one-time activity but is a continuous learning process. As George Bernard Shaw said, “The only man who behaves sensibly is my tailor; he takes my measurements anew every time he sees me, while all the rest go on with their old measurements and expect me to fit them”. Companies need to evaluate the relevance of their CI measures in light of their changing business priorities, and customer expectations. Moreover, they need to assess the intended and unintended consequences of performance metrics. This is particularly important because many companies hardwire performance measures to the incentives such as pay rise or bonus. This creates a target-chasing culture in the organisation. I have seen in many companies how their CI practice maturity scores or financial savings jump up suddenly towards the end of the year to ensure the individuals in the business receive their incentives. Despite all the good intentions behind the performance measures, people inevitably become more focussed on hitting the numbers rather than doing what really matters to the customers and the business. Therefore, it is essential to review and refine the metrics regularly to encourage the right behaviours.
In summary, no one metric rules! Successful companies use a balanced set of metrics and they tie in their CI measures to the desired business outcomes. If the measures don’t work, they learn from their experience, and press the reset button!
Lean Six Sigma Master Black Belt | Elsevier | UK
Originally published on November 18, 2015