Perhaps it is because I spent some time as a freelance project manager and therefore became more likely to be asked to run projects that I had no hand in establishing, that I notice this more now than I used to but it is a problem I have been aware of since I was handed a doomed project by my line manager in the late 1980’s; an experience like riding a tiger, from beginning to bitter end. Even when I was a freelance and able to pick my own jobs, I walked right into another one of them - and I’m still feeling the trauma long after it was finally canned.
So I sat down and asked myself what is going wrong. The symptoms are always the same. The project runs at a madcap pace. The risk and issues registers start long and get longer. Almost every risk that can eventuate does so. The project seems to have no luck. The team works hard, then harder, then becomes sullen and weary. Milestones start to slip and remedial actions don’t seem to work. Effort estimates are ludicrously short of actual effort. You start shedding scope and crashing the schedule but the scope never seems to get smaller. Eventually, you throw in the towel and the customer starts throwing around the blame.
My analysis of a few projects like this suggests it all comes down to one thing: a failure at the very start of the project to estimate it correctly. It all sounds so simple, put like that, but an initial underestimate is a pernicious, systemic problem that surfaces in many different ways, sometimes quite late in the project. An initial underestimate leads to the following inevitable sequence:
1. The overall project plan, based on an initial underestimate, provides inadequate time for each of the major stages of the project: analysis, design, build, integrate, test and deploy. Worse, hard end-dates might be agreed, on which there are external dependencies.
2. Inadequate time for analysis means that the project will discover “hidden scope” during later stages. Some of this hidden scope will not emerge until the build or integration stages.
Hidden scope means that, during the early, or even the middle parts of the project, it can still look as if the project is on track, or, if not quite on track, that it will still complete within the contingency.
3. Inadequate time for design means that there is likely to be re-work as problems crop up during the build, or, more likely, during integration.
4. Less obviously, but just as perniciously, the technical and business analysts on the team, who were unable to grasp the size of the project initially, will continue to compound this error by failing to grasp the size of the effort to completion as the project proceeds. It can take a long time before the project manager is aware of the scale of the underestimation because the senior team members continue to compound their initial error.
5. The result is that the project looks possible until there is a sudden blow-out at the end caused by newly-revealed complexities and major re-work.
6. The project sponsor and the steering committee, who have also been guilty of believing the project is smaller than it really is, are surprised and disappointed when the project manager announces, late in the day, that the project will not meet its deadline after all.
The worst thing about such projects is that standard project management techniques will fail to spot the problem for a very long time. Milestones will be met in the early stages. Initiation, planning and a lot of the project execution can appear to run smoothly before the analysis and design problems begin to surface. Even when major problems have emerged, replanning based on new estimates to completion will fail to save the day because the project team is working with a fundamentally flawed conception of the true size and complexity of the project.
Some approaches, such as rapid application development (RAD), which might be expected to help where there is inherent uncertainty about the scope, can even make matters worse. An RAD approach will delay the detailed analysis and design of what seem to be low-priority parts of the scope. However, when the overall analysis is weak because the effort devoted to it was underestimated, it is quite possible that some of these ‘low priority’ areas contain hidden complexities and hidden dependencies which means that tackling them near the end of the project leaves no time left to resolve the problems they throw up.
It is a common syndrome with a common cause, but what is at the root of such a serious estimating error? If we know that, we can perhaps answer the question; “How does a project manager detect such a project before agreeing to take it on?” Looking across a variety of troubled projects in several different industries, the root cause seems to be that the people involved in setting the budget and the timescale for the project have done so from a position of ignorance. Usually;
- they (and their advisors) have never been involved with a project like this before,
- they (and their advisors) are attempting to force suppliers to deliver to an “aggressive” budget as a cost-saving measure and
- the culture of the organisation is creating a strong pressure on them to hear only what they want to hear about the feasibility of their time and budget constraints.
Q1: Who set the delivery date and on what basis was it chosen?
A1: Ideally, the delivery date should be based on a realistic estimate of the time required to deliver the project, together with an appropriate time contingency. If it was chosen to meet some other external event (the end of a financial year, the start of a new business process that requires the output of this project) or was chosen arbitrarily, or was based on “what seemed reasonable” to the sponsor, the red flag would go up. All is not lost, however, as long as the customer is willing to be flexible about the end date. The answers to questions Q5, Q6 and Q10 now become crucial.
Q2: How was the project scope determined and by whom?
A2: The best answer would be that a thorough analysis of the requirements had been undertaken but this is rarely the case. The best that normally can be hoped for is that the scope was set by the business, working closely with IT specialists, and that it has been clearly and unambiguously documented. If any of these elements are missing (business ownership, IT involvement, and clear scope statement), good answers to questions Q6, Q7 and Q10 become very important.
Q3: How was the project budget determined?
A3: The budget should have been determined based on the needs of the project after the scope, duration and size of the project have been fully understood and an adequate time and effort contingency added. If the budget was set on the basis of a rough estimate, or very little analysis, alarm bells should start ringing. Worse still is the situation where the budget was set before the scope and the project has been sized to fit. In this case, it is very important to have satisfactory answers to questions Q1, Q2 and Q4 and to know that the people involved in the estimating all worked to the fixed budget constraint. A good response to Q7 should also be looked for and, to be safe, Q10.
Q4: How was the staffing of the project determined?
A4: Don’t look just for numbers here but also the right skills and experience. If the answers to Q8 and Q9 are “no” then the chances are good that there could be problems in this area. If the team size and composition was based on estimates that don’t seem to have been soundly based, then you need a positive response to Q10.
Q5: What would happen if the project failed to deliver on time?
A5: Customers don’t like to be asked this question. Either they feel as if you are criticising their ability to deliver, or you are declaring a lack of confidence in your own capabilities. However, it is an issue that must be faced. If there is no willingness to bend on the end-date, or to accept that the initial estimate could be wrong, there may be a problem. There is almost certainly going to be a problem if the answer to question Q1 was not satisfactory. However, if the scope is flexible, a rigid end-date may not be an insurmountable problem. Look out for a good response to question Q6.
Q6: What would happen if the project failed to deliver the full scope?
A6: This can be a very telling question. For a start it will give you an idea of how well the scope is understood. Generally, the more vague the customer is, the more worried you should be. A good response is that all the elements of the scope have been prioritised so that decisions about trade-offs can be made. A bad answer would be that all of the scope is absolutely essential.
Q7: Has a technical lead been appointed and has this person reviewed the feasibility of the project?
A7: I threw this question in as a check on the customer’s commitment to and ownership of the estimate. Sometimes, a customer will rely on third-party estimates (in a tender response, say, or from a consultancy employed to establish the project) and they have a tendency to point the finger at these third parties if the estimates turn out to be wrong. I am always more comfortable with a customer who tells me that they did the estimate themselves, or that they validated the estimate themselves and that they believe in it and will stand by it. I’m even more comfortable if the technical lead who will be responsible for the delivery has validated the estimates him- or herself.
Q8: Has this organisation ever undertaken a similar project before?
A8: If they haven’t, what gives them any confidence in their estimates? If they have, is this estimate in line with their previous experience? More often than not, I am asked to work on projects that are in some ways new to every single stakeholder, including myself. Some customers don’t appreciate that the basis of any estimate is experience — preferably lots of it.
Q9: Has the main supplier ever undertaken a similar project in this industry before?
A9: Very often, suppliers will add significantly to the problem when a project has been underestimated. The very nature of competitive bidding encourages suppliers to take risks. I am looking for two things with this question. Firstly that any part of the estimate based on a supplier’s estimate can be trusted and, secondly, that the customer appreciates the risk of placing trust in an inexperienced supplier — however attractive the low bid is! The necessity of the supplier having experience in the customer’s industry was brought home to me on a project a few years ago where the supplier had extensive worldwide experience in other industries but badly underestimated the effort required to do the same kind of project in a new sector which had complexities they had not experienced before.
Q10: Is there an opportunity to review the project’s foundations before proceeding?
A10: The answer to look for here is, of course, “yes”. And if the customer says “yes” and you have had less than perfect answers from the other nine questions, then take them up on it. Challenge all the assumptions and validate those estimates. Even if you are happy with the customer’s responses, a review consisting only of examining the existing documentation and talking to the key stakeholders, is a prudent thing to do; for your own benefit and for the customer’s.
As project managers, we take on a lot of responsibility for the delivery of projects. When things go wrong we inevitably take the majority of the blame. The set of questions presented above is intended as a tool to help us shed some light on problem projects so we don’t go stumbling blindly into them. Although I’ve described this process as a self-defensive manoeuvre, it should actually be of huge benefit to our customers too. No-one sets out deliberately to run a bad project and a few, well-targeted questions before too many resources have been committed could save our customers, as well as ourselves, from making some terrible mistakes.
No comments:
Post a Comment