If you can't Measure it, you can't Manage it!

There are various proverbs in the general business management world of the form "If you can't Measure it, you can't Manage it!". But is this applicable to the world of Project Management and if so in what ways? I have a firm belief that the answer is yes and in this post I will explain with some examples from projects I have run in the IT domain - but the principles should apply to all other project domains.

Business Case / Benefit Realisation

The Business Case should ideally identify Critical Success Factors (CSF) but these can be difficult to measure on an on-going basis during Project execution. Typically proxy measure(s) can be defined as an indicator towards achievement of the CSF. These are called Key Performance Indicators (KPI) and can be broadly categorised into:

• Lead indicators - these can be measured during project execution
• Lag indicators - these can only be measured after the project has delivered something specific and thus could be post project close-down

The measurement of Lead indicators can be built into Project Plans and assessed as part of Monitoring and Control activities. The measurement of Lag indicators often forms part of the Benefit Realisation Plan, the activities of which often (but not always) only occur post project completion

Task Progress measures

Let us start with measures that you can use with any plan, namely task progress. Whether you collect these measures weekly or more often depends on the project but do consciously consider measurement frequency and whether the task is on the critical path or not! 

My biggest piece of advise here is to avoid the "percentage complete" measure which is not a reliable means of measuring in my experience as illustrated in this cartoon.

Instead look to capture Estimate to Complete (ETC) which is a better measure. I ask for effort (hours) rather than duration because if a person can't spend all day on a task then to state the obvious, the elapsed time will be greater.

Another top tip is to try and enhance your schedule in the area of long tasks (I personally don't like any task greater than 10 days as a rule). In such circumstances, think about whether any intermediate milestones can be captured to provide a more concrete measure of progress rather than ETC. An example with a document drafting task is to understand the sections in advance and measure how many of the total have been completed. So you might have a milestone 50% of sections drafted. The danger here is that there is no measure of the quality but although not perfect as the proverb says "something is better than nothing" :-)

Look for other measures to support task progress 

So you have the basic schedule based measures in place. Now look for other metrics specific to the phase of work being undertaken. Ask the question, what data will give me an independent measure of progress or aid as a check on other measures? Let me give some examples specific to IT system development projects I have run:

• Analysis phase - in a technical analysis migration project I counted the number of objects to analyse and the run rate to achieve the analysis within the planned time for this activity. With these baselines established, measuring the actual objects analysed twice a week gave a means of establishing whether we were on track, ahead or behind plan
• Design - at one client we needed to produce huge end to end design document for a major system with lots of integration to existing components. We agreed the content section by section and split it between about 8 people. I built a specific Excel tracker to monitor sections being authored and delivered to the head architect who did the assembling and quality checks. This was checked daily to ensure we were on track.
• Build phase - from detailed design identify the components to be built and tick them off when they have completed build and unit test. This is a good starting point (e.g. we have completed 40% of the component build) but some components may take more effort than others. This is where a build sheet can be used to measure whether we are on track to complete in the planned time with the agreed component list and available resource availability. I wrote a specific post on this "build sheet" approach
• Test phase - there are lots of measures to look for! In terms of pure execution, establish with your test manager the "test case run plan", i.e. how many test cases should be executed (not necessarily passed) over time. In simplistic terms if I have (say) a 5 week test phase, I will look for a plan to achieve execution of all test cases at least 1 week before the end of the phase as you should anticipate some residual defects to fix and retest. I will also have metrics reported ideally daily around number of open defects at various statuses (raised but nothing yet done with, in analysis, being fixed, ready for retest etc). You need to look for being "over the open defect curve hump" ideally by two thirds into the test phase

So here are some examples I have used and whether you agree / disagree is not the point, I recommend that you to look for supporting progress metrics in your particular project circumstances.

Financial measures

The basic financial measure is "burn rate" - how much money is the project burning per week / month? This can vary depending on where you are in the project lifecycle as more resources are added or rolled off. So understand what you originally budgeted and what you are actually burning to see whether the project in on track to deliver under, on or over its budget.

Conclusions

I have hopefully illustrated the benefit of quantifiable measurements across many aspects of your Project execution whether progress monitoring, benefit monitoring or financial monitoring. But don't just take my word for it! Galileo said:

Measure what is measurable, and make measurable what is not so

I've just re-read an old post on Monitoring and Control and it covers similar ground. This proves something - it could be that my memory is failing or that this is a very important topic which is at the forefront of my brain. I like to think that it is the latter!