I follow Seth Godin as he has incredible insight and ability to simplify things down to a few key words. His post, Speedometer confusion is one of may such posts.
Too often, being busy, is confused with productive. We set metrics like how many defects found, how many test cases run and how many hours tested. But what exactly do these metrics mean to software quality? Looking down at our speedometer, we may think we travelling at 200km/hour for the last 8 hours covering some 1600km. But what if when we check the GPS, it shows we only covered 100km.
We are often wary of establishing effective KPI, worried about what these would really reveal, or worried management may not understand. Opting to track “safe” KPI that show some form of effort rather than effect. Rather than number of test cases, why not measure percentage coverage? Rather than defects found, why not measure defects missed? Rather than hours spent testing, hours saved?
It is nearly impossible to improve, if one fails to take an accurate measure of the current situation and set goals for the desired outcome.