Measurements gone bad

One of the process areas of CMMI is Measurement and Analysis.

You can go and read the model as to what it is specifically but here is the short of it:  define measures that meet your information needs.  And use them.

Unfortunately, in life and business, I see use of measures that make absolutely NO SENSE to your information needs.  Let me provide you with some examples.

This morning I was meeting with a colleague of mine.  He is preparing for a competition in amateur body building.  He is pretty competitive in his class.  We often share conversations about process models (like CMMI), people, tools and athletics.  As he is a body builder and I am a runner, there are many things we do differently but there are many principled things we do the same.

For example, we both deal with the principle of a taper, but our approaches at a technical level are markedly different in how we do that.  The mental discipline however to effectively get through the taper is very similar. 

We also deal with a variety of measures throughout our training.  For him, it is measures regarding size of muscles, body weight, percent body fat, how much weight he pushing in a workout, how many calories he is eating, etc.  For me, it is about the miles I have put in, the pace I hit in a workout, the HR I managed over a workout, body weight, days at altitude, number of feet climbed (note – these are not comprehensive lists but they give you an idea).

We use these data to make informed decisions about how our training has progressed and in which direction we ought to take it.  The data serves as a dashboard of measures.  At any point in the training cycle, one element of the dashboard may be slightly out of whack as we look to refine the system for more optimal performance.  We iterate through these measures, consider them, recalibrate them, change their expectations and may add or eliminate certain ones.

But ultimately, they help drive to a singular goal – peak performance on a specific event day.  For me a race, for him a posing competition.

As I was asking him about his upcoming competition, and how lean he currently is, he noted how his BMI (body mass index) will often categorize him as overweight, or even obese.  I can tell you that any body that can see would never come to such a conclusion. 

But we have this measure called BMI that is used to calculate such things and in many places it is used.  Does not make any information sense, but hey – it works for a lot of folks so it has to work here too.

Clearly stupid.

Similarly in business I see the use of measures in a one size fits all type approach.  Ask any engineer about their thoughts on defects per KLOC and they will probably give you an earful.

Now this is NOT to say that everybody gets to define any measure they want for their needs when they want.  In fact, some level of  standardization is completely relevant.  In a race, it the time the clock says at the end of the race.  In a posing competition, it is what the judges rank you.  For most businesses it ends up as a bottom line on your financials.  Some level of standardization across a business makes sense for the purposes of clarity in speaking to each other and your customers.