There are a lot of pitfalls with metrics, they can seem like a magic bullet to control the output of a team. Measurement is certainly useful but how you use it is critical to ensuring your projects succeed.
Let’s take an example. I tell my teams that I’m measuring their success on defects raised. For the testers their success is the more defects they find the better, for the developers the less defects found the better. I have now introduced an extrinsic reward associated with a metric. It is highly likely that the developers don’t want defects in their system and the testers were perfectly happy when they didn’t find any but that’s about to change….
Now comes the unintended consequences. Testers start to raise defects for almost anything such as a text box not quite large enough (in their opinion) etc. This annoys the developers who could just fix the defect really quickly and don’t need it raised, slowing the team down. The developers figure that they’ll not get the software in front of the testers quickly, instead they’ll sit on it for days trying things out, increasing the cycle time. Then the developers then figure that writing less software is the best way for less defects to be found and slow down. The testers and developers and now effectively at war and will fight about whether something is a change or a defect.
This as we can see can be disastrous. So how do we fix this issue. Firstly let’s trust the team to do the best they can. Secondly let’s use the metrics to help them improve and don’t reward them externally. For example we can give the team the metrics and ask them to work out where they can improve. This gives them the information they need without attaching a reward to it, people usually want to do a good job. If you have someone monitoring metrics in this way consider moving them into the team so that they are part of the delivery. They are now as accountable for quality as the rest of the team and inform them that they are there to help.
Did you already mess this up? Here’s a way of understanding that, let’s evaluate our current metrics by asking our teams:
– What was the metric originally meant to be for?
– What is it being used for now?