Ten Essential Questions for Project Managers

As a project manager with almost twenty years’ experience under my belt, I have, like most people in this profession, seen some spectacular stuff-ups. In fact, as project managers, our job often seems to be to play the starring role in these never-ending nightmares. Yet most of us know that, while a good project manager is able to pull some impending disasters out of the fire, many projects were just set up to fail and there is nothing even the best project manager can do except ensure that the customer’s money doesn’t haemorrhage too badly.

Perhaps it is because I spent some time as a freelance project manager and therefore became more likely to be asked to run projects that I had no hand in establishing, that I notice this more now than I used to but it is a problem I have been aware of since I was handed a doomed project by my line manager in the late 1980’s; an experience like riding a tiger, from beginning to bitter end. Even when I was a freelance and able to pick my own jobs, I walked right into another one of them - and I’m still feeling the trauma long after it was finally canned.

So I sat down and asked myself what is going wrong. The symptoms are always the same. The project runs at a madcap pace. The risk and issues registers start long and get longer. Almost every risk that can eventuate does so. The project seems to have no luck. The team works hard, then harder, then becomes sullen and weary. Milestones start to slip and remedial actions don’t seem to work. Effort estimates are ludicrously short of actual effort. You start shedding scope and crashing the schedule but the scope never seems to get smaller. Eventually, you throw in the towel and the customer starts throwing around the blame.

My analysis of a few projects like this suggests it all comes down to one thing: a failure at the very start of the project to estimate it correctly. It all sounds so simple, put like that, but an initial underestimate is a pernicious, systemic problem that surfaces in many different ways, sometimes quite late in the project. An initial underestimate leads to the following inevitable sequence:

1. The overall project plan, based on an initial underestimate, provides inadequate time for each of the major stages of the project: analysis, design, build, integrate, test and deploy. Worse, hard end-dates might be agreed, on which there are external dependencies.

2. Inadequate time for analysis means that the project will discover “hidden scope” during later stages. Some of this hidden scope will not emerge until the build or integration stages.
Hidden scope means that, during the early, or even the middle parts of the project, it can still look as if the project is on track, or, if not quite on track, that it will still complete within the contingency.

3. Inadequate time for design means that there is likely to be re-work as problems crop up during the build, or, more likely, during integration.

4. Less obviously, but just as perniciously, the technical and business analysts on the team, who were unable to grasp the size of the project initially, will continue to compound this error by failing to grasp the size of the effort to completion as the project proceeds. It can take a long time before the project manager is aware of the scale of the underestimation because the senior team members continue to compound their initial error.

5. The result is that the project looks possible until there is a sudden blow-out at the end caused by newly-revealed complexities and major re-work.

6. The project sponsor and the steering committee, who have also been guilty of believing the project is smaller than it really is, are surprised and disappointed when the project manager announces, late in the day, that the project will not meet its deadline after all.

The worst thing about such projects is that standard project management techniques will fail to spot the problem for a very long time. Milestones will be met in the early stages. Initiation, planning and a lot of the project execution can appear to run smoothly before the analysis and design problems begin to surface. Even when major problems have emerged, replanning based on new estimates to completion will fail to save the day because the project team is working with a fundamentally flawed conception of the true size and complexity of the project.

Some approaches, such as rapid application development (RAD), which might be expected to help where there is inherent uncertainty about the scope, can even make matters worse. An RAD approach will delay the detailed analysis and design of what seem to be low-priority parts of the scope. However, when the overall analysis is weak because the effort devoted to it was underestimated, it is quite possible that some of these ‘low priority’ areas contain hidden complexities and hidden dependencies which means that tackling them near the end of the project leaves no time left to resolve the problems they throw up.

It is a common syndrome with a common cause, but what is at the root of such a serious estimating error? If we know that, we can perhaps answer the question; “How does a project manager detect such a project before agreeing to take it on?” Looking across a variety of troubled projects in several different industries, the root cause seems to be that the people involved in setting the budget and the timescale for the project have done so from a position of ignorance. Usually;

• they (and their advisors) have never been involved with a project like this before,
• they (and their advisors) are attempting to force suppliers to deliver to an “aggressive” budget as a cost-saving measure and
• the culture of the organisation is creating a strong pressure on them to hear only what they want to hear about the feasibility of their time and budget constraints.

As a result of this analysis I have developed a short questionnaire which I now use to screen out problem projects. I have restricted myself to ten questions so that they are few enough to be included quite naturally in an interview with prospective customers. The ten questions are listed below with what I believe would be acceptable answers.

Q1: Who set the delivery date and on what basis was it chosen?

A1: Ideally, the delivery date should be based on a realistic estimate of the time required to deliver the project, together with an appropriate time contingency. If it was chosen to meet some other external event (the end of a financial year, the start of a new business process that requires the output of this project) or was chosen arbitrarily, or was based on “what seemed reasonable” to the sponsor, the red flag would go up. All is not lost, however, as long as the customer is willing to be flexible about the end date. The answers to questions Q5, Q6 and Q10 now become crucial.

Q2: How was the project scope determined and by whom?

A2: The best answer would be that a thorough analysis of the requirements had been undertaken but this is rarely the case. The best that normally can be hoped for is that the scope was set by the business, working closely with IT specialists, and that it has been clearly and unambiguously documented. If any of these elements are missing (business ownership, IT involvement, and clear scope statement), good answers to questions Q6, Q7 and Q10 become very important.

Q3: How was the project budget determined?

A3: The budget should have been determined based on the needs of the project after the scope, duration and size of the project have been fully understood and an adequate time and effort contingency added. If the budget was set on the basis of a rough estimate, or very little analysis, alarm bells should start ringing. Worse still is the situation where the budget was set before the scope and the project has been sized to fit. In this case, it is very important to have satisfactory answers to questions Q1, Q2 and Q4 and to know that the people involved in the estimating all worked to the fixed budget constraint. A good response to Q7 should also be looked for and, to be safe, Q10.

Q4: How was the staffing of the project determined?

A4: Don’t look just for numbers here but also the right skills and experience. If the answers to Q8 and Q9 are “no” then the chances are good that there could be problems in this area. If the team size and composition was based on estimates that don’t seem to have been soundly based, then you need a positive response to Q10.

Q5: What would happen if the project failed to deliver on time?

A5: Customers don’t like to be asked this question. Either they feel as if you are criticising their ability to deliver, or you are declaring a lack of confidence in your own capabilities. However, it is an issue that must be faced. If there is no willingness to bend on the end-date, or to accept that the initial estimate could be wrong, there may be a problem. There is almost certainly going to be a problem if the answer to question Q1 was not satisfactory. However, if the scope is flexible, a rigid end-date may not be an insurmountable problem. Look out for a good response to question Q6.

Q6: What would happen if the project failed to deliver the full scope?

A6: This can be a very telling question. For a start it will give you an idea of how well the scope is understood. Generally, the more vague the customer is, the more worried you should be. A good response is that all the elements of the scope have been prioritised so that decisions about trade-offs can be made. A bad answer would be that all of the scope is absolutely essential.

Q7: Has a technical lead been appointed and has this person reviewed the feasibility of the project?

A7: I threw this question in as a check on the customer’s commitment to and ownership of the estimate. Sometimes, a customer will rely on third-party estimates (in a tender response, say, or from a consultancy employed to establish the project) and they have a tendency to point the finger at these third parties if the estimates turn out to be wrong. I am always more comfortable with a customer who tells me that they did the estimate themselves, or that they validated the estimate themselves and that they believe in it and will stand by it. I’m even more comfortable if the technical lead who will be responsible for the delivery has validated the estimates him- or herself.

Q8: Has this organisation ever undertaken a similar project before?

A8: If they haven’t, what gives them any confidence in their estimates? If they have, is this estimate in line with their previous experience? More often than not, I am asked to work on projects that are in some ways new to every single stakeholder, including myself. Some customers don’t appreciate that the basis of any estimate is experience — preferably lots of it.

Q9: Has the main supplier ever undertaken a similar project in this industry before?

A9: Very often, suppliers will add significantly to the problem when a project has been underestimated. The very nature of competitive bidding encourages suppliers to take risks. I am looking for two things with this question. Firstly that any part of the estimate based on a supplier’s estimate can be trusted and, secondly, that the customer appreciates the risk of placing trust in an inexperienced supplier — however attractive the low bid is! The necessity of the supplier having experience in the customer’s industry was brought home to me on a project a few years ago where the supplier had extensive worldwide experience in other industries but badly underestimated the effort required to do the same kind of project in a new sector which had complexities they had not experienced before.

Q10: Is there an opportunity to review the project’s foundations before proceeding?

A10: The answer to look for here is, of course, “yes”. And if the customer says “yes” and you have had less than perfect answers from the other nine questions, then take them up on it. Challenge all the assumptions and validate those estimates. Even if you are happy with the customer’s responses, a review consisting only of examining the existing documentation and talking to the key stakeholders, is a prudent thing to do; for your own benefit and for the customer’s.

As project managers, we take on a lot of responsibility for the delivery of projects. When things go wrong we inevitably take the majority of the blame. The set of questions presented above is intended as a tool to help us shed some light on problem projects so we don’t go stumbling blindly into them. Although I’ve described this process as a self-defensive manoeuvre, it should actually be of huge benefit to our customers too. No-one sets out deliberately to run a bad project and a few, well-targeted questions before too many resources have been committed could save our customers, as well as ourselves, from making some terrible mistakes.

A Theory of Human-Computer Interaction

The Trouble With HCI

Human-Computer Interaction (HCI) as a discipline is drifting. It has no long-term objectives and no strategic direction. This would be forgivable if, like other disciplines, it had methods which were widely accepted as being likely to produce increases in understanding. But HCI has no such methods. It is not possible to say that HCI is a science, it is barely possible to describe it as an engineering discipline and yet we, as practitioners, are reluctant to relegate it to the status of a craft.

Part of the difficulty appears to be that it is hard to identify the “core” subject-matter of HCI. This is often acclaimed by practitioners as a benefit because it leads to the need for a multi-disciplinary approach which is considered to be a good thing. Yet without an identifiable subject-matter, it is difficult to identify what are the important issues in HCI, it is hard to say what an appropriate method for studying or practicing HCI would be and it is impossible to develop a theory of HCI.

This last is of peculiar importance. If we could produce a theory of HCI (whatever that might be), it would at the same time tell us what HCI is (as defined by the theory) and what its scope is. If we could say what the core subject-matter of HCI was, we would, at the same time, be proposing something like a theory of HCI. The problem of finding a theory of HCI is clearly what Rittel and Webber called a “wicked problem”. That is, finding a way to understand the problem is tantamount to finding a way to solve it. The way to tackle the problem then is to break the vicious circle by creating a theory of HCI. This would then seed a process of elaboration and refutation which would eventually end in there being a theory (or set of theories) of HCI which were acceptable to practitioners in the field—almost regardless of the quality of the original.

What Would a Theory of HCI Look Like?

The definition of a theory used here is a little idiosyncratic but it is one that should seem familiar to HCI people. I would like to propose that a theory is the set of rules that defines the behaviour of a system of conceptual objects and relationships. The set of conceptual objects and relationships itself, I will call a conceptualisation and we can imagine that there could be an arbitrary number of theories for any particular conceptualisation as well as there being an arbitrary number of conceptualisations for any field.

A Conceptualisation

A conceptualisation belongs to a field just as a theory does. It describes in detail the objects that are important to the field and the relationships between them. In the field of HCI we could imagine a conceptualisation involving objects such as “user”, “computer”, “input device”, “data” and so on, and relationships between them such as “display”, “use”, “enter”, etc.. Yet there is an arbitrary number of other conceptualisations for the same field. If one were to read the HCI literature and attempt to extract the conceptualisation used by the author in each separate paper, a large set of more-or-less complete, vague, overlapping and coherent conceptualisations would be the result.

To some extent, it will be possible to say that some conceptualisations are “better” than others and we can imagine dimensions along which to judge them such as completeness of coverage of the field, the degree of discrimination or granularity of the concepts and the extent to which practitioners in the field will agree with the conceptualisation. However, it is also true that some conceptualisations may be better than others for different purposes and that many may be equally good for the same purpose. Nevertheless, I feel that the lack of a clearly articulated common conceptualisation for HCI is a significant problem for the field. It hinders communication within and outside the community of practitioners and it effectively blocks the development of theory.

A Theory

A theory is the set of rules that governs the behaviour of the system described by the conceptualisation. In Newtonian mechanics, for instance, the three laws of motion are examples of such rules.

It is unlikely that a theory of HCI would ever have the simple elegance of a physical theory. The conceptualisation of HCI is, to begin with, far more complex than the conceptualisation of space and mass. Additionally, the individual concepts are themselves considerably more complex than those found in physics. Contrast typical HCI concepts such as “user”, “display” and “message” with concepts from Newtonian physics such as “mass”, “distance” and “time”. Nevertheless, it is still conceivable that such a theory could be built—even if the rules were far more elaborate and commonly involved the use of qualification and uncertainty.

Repositioning Usability

Anyone who has worked in the field of user inteaction design will know what a hard sell it is in all but the most enlightened of organisations. IT departments and IT services companies are often the most resistant customers. Why is that?

Well, I believe that, after 25 years in the business, I've finally worked it out. Below is the abstract and conclusion from a paper I've just written on the subject. The full text can be found at:

http://graham.storrs.cantalibre.com/compulsion/repositioningusability.html

ABSTRACT
In the IT services industry the various specialisms which deal directly with usability are generally considered of little value – in fact, of so little value that they are dispensed with entirely in the great majority of development projects. Some efforts have been made to cost-justify usability but IT suppliers and customers remain unconvinced. This discussion paper argues that the main reason for the perception of the low value of usability is because it is incorrectly regarded as a property of IT systems. The paper argues that a more realistic view- and one that accords better with best practice in the field - is that usability is a property of business processes. In particular it characterises the quality of the communication of the people participating in these process with the tools, equipment and media they use to assist them.
CONCLUDING REMARKS
It is clear to almost everyone that the software services industry is not doing a good job for its customers. It is also clear that, despite some inroads into product development, usability is not thought to offer much of value, even to an industry in such disarray. I contend that this is because the services which usability professionals are offering tend to have very low intrinsic value except in the area of task redesign. Task redesign, I argue, is the same as business process redesign. In this area, usability professionals have a huge contribution to make because process engineers have largely neglected the issue of process usability. Usability professionals have developed an extensive collection of task analysis and design approaches that help ensure process usability and these methods and techniques could profitably be adopted in industry. Beyond even this, the use of iterative prototype and evaluate cycles to create usable processes, has the potential greatly to improve process quality. Finally, I have argued that, to make use of these methods and techniques, organisations must radically change the way they procure IT projects. Specifically, they should fully specify their business processes, down to the level of a detailed interaction design, which would then form the basis of an IT procurement.