A survey of information systems development project performance.

Wright, Keith ; Capps, Charles

CURRENT INFORMATION TECHNOLOGY (IT) ISSUES

The rate of information systems development project failure in the 1980s and 1990s was routinely documented to be above 50%, the larger the development; the more likely it was unsuccessful (SIMPL & NZIER, 2000). A 1994 study of IS development projects in the British public sector estimated that 20% of expenditures were wasted, and a further 30% to 40% did not produce perceivable benefits (Wilcocks, 1994). Also in 1994, the U.S. General Accounting Office reported that spending of more than US$200 billion in the previous twelve years had led to few meaningful returns. A 1995 study of over 8,000 IS projects by Johnson revealed that only 16% were completed on time and within budget (Johnson, 1995). The U.S. Internal Revenue Service, with an annual computer budget of US$8 billion, managed "a string of project failures that have cost taxpayers $50 billion a year [mainly defined as revenue forgone]--roughly as much as the yearly net profit of the entire computer industry" (James, 1997). Collins and Bicknell (1997) estimated that public sector failures in the United Kingdom cost 5 billion [pounds sterling]. The Wessex Health Authority's Regional Information Systems Plan was cancelled after more than 43 million [pounds sterling] had already been spent, with little achieved (Collins et al., 1997). The New Zealand Police abandoned an IS development in 1999, at a cost of more than NZ$100 million, after years of development provided little more than an e-mail system and a number of terminals run by a 1970s-era mainframe. A study by SIMPL & NZIER (2000) found that the success rate was only 55% for projects under US$750,000; however, for those with budgets over US$10 million, no projects were successful. A 2001 U.S. Standish Group survey of IS projects found success rates were as follows: 59% in the retail sector, 32% in the financial sector, 27% in manufacturing, and 18% in government. Overall, the average success rate was 26%. In all, 46% of the projects had problems, including being over budget, behind schedule, or delivered incomplete. Another 28% failed altogether or were cancelled, and cost overruns averaged nearly 200% (SIMPL & NZIER, 2000).

The beginning of the 21st century showed little improvement in IS development project performance. In 2002, the United Kingdom's National Health Service initiated the largest-ever public sector project at an estimated cost of 11 billion [pounds sterling]. This led to the introduction of new information systems in almost every hospital, but it was still considered a failure (Rainer & Turban, 2009). A benefit payment scheme involving the British Post Office, the Department of Social Security, and the computer company ICL was abandoned after three years and a cost of 300 million [pounds sterling] (The Economist, 2002). An already obsolete air-traffic support system opened at Swanson in the United Kingdom, in 2002, six years late and 180 million [pounds sterling] over budget (The Economist, 2002). The Canadian Firearms Program increased from initial estimates of C$113 million to more than C$1 billion, an overrun of almost 900% (Auditor, 2002). Georgiadou (2003) found five out of six corporate projects are considered unsuccessful, with one-third cancelled. Of the two-thirds that were not cancelled, price and completion times were almost twice what had originally been planned. Dalcher and Genus (2003) reported US$150 billion was wasted per annum on information and communications technology failures in the public and private sectors in the United States and US$140 billion in the European Union. A 2004 Standish Group report estimated a success rate of 29%, with 53% of the projects having problems, and a failure rate of 18%. The Royal Academy of Engineering and the British Computer Society (2004) found that 84% of public sector projects resulted in failure. Vast sums of money, mostly provided by aid agencies, have been spent on health and other information systems in South Africa, on donor-funded IS projects in China, and on World Bank funded projects in Africa. Overwhelmingly, these projects have resulted in varying degrees of failure (Heeks, 2002, 2004). The Standish Group's 2009 report showed only 32% of all projects succeeded, while 44% were problematic and 24% failed. In the United States the best known IS project failures are the FBI Trilogy Project (Knorr, 2005; US GAO, 2006), the California Motor Vehicles Driver Licensing System (Bozman, 1994), and the Denver airport baggage handling system (Montealegre & Keil, 2000).

In summary, the consensus is that 20% to 30% of all IS development projects are perceived as overwhelming failures, while 30% to 60% are partial failures (Collins et al., 1997; Corner & Hinton, 2002; Georgiadou, 2003; Heeks, 2002; Iacovou, 1999; James, 1997).

IS projects that run wildly over time and budget have been labeled "runaways" (Glass, 1998; Mann, 2003). Runaway projects have been said to take on a life of their own without adding business value (Zmud, 1980; DeMarco, 1982; Abdel-Hamid & Madnick, 1991; Johnson, 1995; Mahring & Keil, 2008). One of the main reported causes of "runaway" projects has been said to be management over- commitment to the project (Keil, 1995). The management behavior that underlies runaway projects resembles what IS researchers have called the process of "escalation of commitment to a failing course of action" (Brockner, 1992; Keil, 1995; Schmidt & Calantone, 2002). The reverse of this process has been defined as IS project de-escalation (Keil & Robey, 1999; Montealegre & Keil, 2000; Royer, 2003; Heng et al., 2003). The literature suggests four general types of determinant factors of project commitment: project, psychological, social, and organizational. (For a good review see Newman & Sabherwal, 1996). The tendency for managers to over-commit to an obviously failing project has lent credence to the strategy that it is sometimes better to radically change the entire organization rather than incrementally improve the immediate project team (Pan et al., 2006a, 2006b). This line of research stems from punctuated equilibrium models that have their roots in biology (Eldredge & Gould, 1972) and are increasingly being proposed in the organizational sciences (e.g., Tushman & Anderson, 1986; Mokyr, 1990; Gersick, 1991).

Most of the research on IS project failure was conducted prior to the age of IT governance practices put in place after the 2002 Sarbanes Oxley Act. IT governance implies a management system in which top management has a much larger influence on IS development project decision-making. The exploratory research presented here responds to the need for an improved understanding of IS project performance in the post-Sarbanes Oxley environment. The primary purpose of this study was to determine the extent to which, given recent increase in IT governance practices, runaway projects continue to be common and to help determine promising new areas for future research in preventative measures. Another purpose of the study was to determine if, as predicted by punctuated equilibrium theory, radical organizational changes are more effective in withdrawing commitment to runaway projects than are incremental measures.

The remainder of the paper is organized as follows. Next is a review of the literature around the social and psychological causes of "runaway" projects. Then we present our research method followed by a discussion of the results conclusions.

SOCIAL AND PSYCHOLOGICAL CAUSES OF "RUNAWAY" PROJECTS

Management literature identifies four general types of determinantal factors of project commitment: project, psychological, social, and organizational (Newman & Sabherwal, 1996; Keil & Robey, 1999; Pan et al., 2006; Hirschheim, Klein, & Newman, 1991). Project factors include the costs and benefits as perceived by management. Projects are considered prone to over-commitment when they involve a large potential payoff, when they require a long-term investment before substantial gain, and when setbacks are perceived as temporary surmountable problems (Keil, 1995; Keil, Man & Rai 2000).

Psychological factors cause managers to believe the project will eventually be successful (Brockner, 1992). Contributing causes include the manager's previous experience, the degree to which the manager feels personally responsible for the outcome (Newman & Sabherwal, 1996), and cognitive biases (Tversky & Kahneman, 1981). Other research has suggested that managers may engage in a kind of "self-justification" behavior committing additional resources to a project instead of ending it and admitting that their earlier decisions were flawed (Whyte, 1986; Staw & Ross 1987; Ross & Staw, 1993). This line of research, known as self-justification theory (SJT), is grounded in Festinger's (1957) theory of cognitive dissonance. Prospect theory focuses on the cognitive biases that influence human decision- making under uncertainty. A derivative of this theory is the so-called "sunk cost" effect in which decision makers exhibit a tendency to "throw good money after bad" (Garland & Conlon, 1998). That research suggests that sunk costs may influence decision makers to adopt a negative frame, promoting risk-seeking (IS project escalation) behavior. Approach-avoidance theory suggests there is a natural tendency for management to over-commit to IS projects because of the "completion effect," which suggests the motivation to achieve a goal increases as an individual gets closer to that goal (Garland & Conlon, 1998; Pan et al., 2006). The completion effect is particularly relevant to software projects, which frequently exhibit the so-called "90% complete" syndrome (DeMarco, 1982; Garland & Conlon, 1998).

Social factors also promote IS project over-commitment (Newman and Sabherwal, 1996; Pan et al., 2006). Social factors include competitive rivalry with other social groups, the need for external justification, and norms for consistency (Brockner et al., 1979; Hirschhem, Klein, & Newman, 1991). Projects are prone to over-commitment when competitive rivalry exists between the decision-making group and another social group, when external stakeholders believe the project will be successful, or when norms of behavior favor persistence (Ross & Staw, 1993). One social determinant of commitment is "the desire not to lose face or credibility with others" (Staw & Ross 1987). The concept of "face saving" is grounded in self-justification theory (Whyte, 1986; Staw & Ross, 1987; Ross & Staw, 1993) and is discussed in the IS project de-escalation literature (Montealegre & Keil, 2000).

Organizational factors in project over-commitment include the structural and political factors that form the "ecosystem" of a project. Information systems researchers (e.g., Keil, Mann & Rai, 2000) have referred to several other organizational factors that may affect commitment to an IS project: top management's knowledge of information technology (Vitale et al., 1986), information intensity of the organization's value chain (Johnston & Carrico, 1988), and the maturity of the IS function (Sabherwal & King, 1992). Other organizational factors identified in the literature include top management support for the project, administrative inertia in the organization, the extent of project institutionalization, and the extent to which it is perceived as strategic (Johnston & Carrico, 1988).

More recent work on IT project de-escalation suggests projects may be resistant to de-escalation because long periods of organizational "equilibrium" periods are difficult to disrupt because of the constancy of the "deep structure" of the organization (Gersick, 1991; Pan et al., 2006; Orlikowski, 1993). As a result, organizations may be unable to change substantially unless forced by a radical organization change or other crisis (Tushman & Romanelli, 1985; Sastry, 1997). In a number of industries, it has been observed that long periods of unsuccessful "incremental" organizational change tend to be interrupted by short periods of radical change called "revolutionary periods" (Abernathy & Utterback, 1978; Utterback & Suarez, 1993). This pattern termed "punctuated equilibrium," was originally identified in the biological sciences (Eldredge & Gould, 1972) and was subsequently adopted in the management literature (e.g., Tushman & Anderson, 1986; Mokyr, 1990). According to punctuated equilibrium theory, organizations tend toward "equilibrium" because of the permanence of the organization's "deep structure." The deep structure of an organization consists of its "alliances, associations and co operations with interlocking interests" (Anderson & Tushman, 1990). This analysis complements Tushman and Romanelli's (1985) identification of "performance pressures ... whether anticipated or actual" as the fundamental agents of organizational reorientation. Tushman, Newman, & Romanelli (1986) described the scenario of an organization falling into serious trouble before responding by replacing its top management as typical. They found "externally recruited executives are more than three times more likely to initiate frame-breaking change than existing executive teams ... Failures caused by inappropriate deep structures are destined to elude the (misdirected) efforts of current system members to correct them. Unless such failures kill the system, they command increasing attention and raise the likelihood that newcomers will either be attracted or recruited to help solve the problems. The newcomer has the opportunity to see the system in an entirely different context than incumbent members, and he or she may begin problem solving on a new path" (Tushman, Newman, & Romanelli, 1986).

The preceding review demonstrates the extreme complexity of the study of information system (IS) development project performance. Existing research demonstrates the presence of many important factors that affect such a project. This raises the question of the most important areas for further IS practice research: Should the focus be on the subtleties of the various indirect psychological and social factors mentioned above? Or, is it more practical to concentrate on the practice of project management itself? The answers to these questions are the impetus for this study.

RESEARCH METHOD

In order to address these research questions concerning IS development projects a simple questionnaire was developed to survey IT audit professionals. The design of the survey was based on Dillman's (1978) "total design method." Auditors were selected for the study because they are likely to be more objective than other IS project stakeholders, such as managers directly responsible for problematic projects. The sample was designed to select professional IT auditors who would be most likely to be involved in information systems development. The pool of individuals represented approximately one thousand (1000) Information Systems Audit and Control Association (ISACA) members in the Houston, Texas area. The survey was designed to gather data concerning the perceived frequency of problematic IS development projects, the perceived major sources of problems, and the recommended magnitude of corrective actions. A survey is usually the most cost-effective way of collecting data on a large number of IS projects (Mann, 1996). To get a reasonable response rate, the survey was designed to be completed in less than ten minutes. To ensure the survey questions demanded minimal cognitive effort, questions consisted mostly of paired category choices.

The pool of one thousand auditors (1000) was emailed a link to the survey that was posted on the international ISACA web site. The approximately one hundred (100) respondents were asked to consider projects with which they were familiar that fit the definitions of "problematic," "runaway," and "failed" on the survey form. The survey had five sections: introduction, demographics, project performance, corrective actions, and closing. Please see Appendix A for the actual survey questions. The survey was refined through two iterations of pre-testing.

The demographics of the sample are shown in Tables 1, 2, 3 and Table 12 below. Almost half the respondents reported having more than fifteen years experience, while over 40% reported having more than five years experience. Almost 80% reported having the Certified Information Systems Auditor (CISA) certificate; and, about half reported they had been involved with either a problematic or a runaway project (Table 12).

Variables, such as auditor experience, could have affected the results. However, a similar survey of IT auditors showed these variables did not affect overall results (Keil et al., 2000). Thus, to keep the survey brief, we chose not to control for these factors. Furthermore, as was the case in Keil et al. (2000), our method did not allow us to measure non-response bias. To increase the reliability of the survey instrument, three different measures of project performance were operationalized: (1) "failed" projects, (2) "problematic" projects and (3) "runaway" projects

RESULTS AND DISCUSSION

OVERALL PROJECT PERFORMANCE

The results for overall project performance are shown in Table 4. Significantly more of the respondents reported the overall rate for both problematic and failed projects are above 50%. These high frequencies are corroborated by the fact that 40% of the respondents reported having been involved with a "runaway," and 60% reported involvement with a "problematic" project (See Table 12). The findings regarding "runaways" reported here appear to contrast sharply with researchers who contend that software project runaways are rare events (Glass, 1997).

SOURCES OF PROJECT PROBLEMS

As can be seen in Tables 6 and 7, the most important sources of problems for both problematic and runaway projects were perceived to be formal attributes of the project team such as size, skills, etc., rather than informal social or psychological attributes of project stakeholders.

GOVERNMENT VERSUS PRIVATE SECTOR PERFORMANCE

Table 8 shows significantly more respondents believed that all three project types--failed, problematic, and runaway--are more likely to occur in government than industry. We argue that this result is consistent with that predicted by punctuated equilibrium theory (the argument is presented as an informal theorem in Table 7).

THE ROLE OF "COLLECTIVE BELIEF" IN MANAGEMENT OVER-COMMITMENT

Another important determinant of management commitment to an IS project has been termed the "collective belief" in the eventual success of the project (Royer, 2002). Our results suggest that government managers seem less confident in their abilities than do private sector executives (See Table 8, question 2.2) and that government is more likely to produce runaways and failures than is the private sector (Table 9, question 3.11). However, these results run counter to that reported in Royer (2002). There it was found that "collective belief" in the eventual success of the project was a primary determinant of management over-commitment.

CORRECTIVE ACTIONS, INCREMENTAL OR RADICAL?

Survey section three (corrective actions) was designed in part to measure whether, as predicted by punctuated equilibrium theory, respondents thought it more effective to apply radical (aka "revolutionary") measures to turn around "runaway" projects than to apply more moderate (aka "incremental") ones (Gersick, 1991). These results are consistent with the punctuated equilibrium argument presented as an informal theorem in Table 9: for "runaway" projects, question 3.8 (See Table 11) respondents believed it more beneficial to choose the radical action of replacing project management rather than the incremental action, simply educate them. However, for "problematic" projects, the respondents reported the opposite conclusion: that an incremental change (i.e., education) is more beneficial than the more radical change, replacing staff (See Tables 11 and 12). Overall though, our results strongly support the conclusion that project factors are more important than organizational factors in determining project failure or success.

CORRECTIVE ACTIONS, TOP MANAGEMENT OR MIDDLE MANAGEMENT?

In all cases, our respondents believed it more beneficial to take corrective action at the project management level than at the top level of management (See Tables 11 and 12). These results suggest the most important causes and remedies for poor project performance have to do more with project management practices than with the structure of the surrounding organization. This does not support the notion that IT governance initiatives introduced by many large organizations early in the 21st century have improved performance of IS development projects.

RISK AVERSION IN RUNAWAY PROJECTS

Another purpose of the study is to help managers decide on an appropriate level of risk aversion in "runaway" projects. On survey question 3.12 (See Table 4) respondents reported an insignificant difference between those that believe managers prematurely kill runaway projects less than 20% of the time, and those that believe mangers prematurely kill runaway projects more than 20% of the time. Thus, we speculate that managers actually kill projects prematurely about 20% of the time. How managers feel about this likelihood is important to the organization's bottom line. For example, suppose management decides to kill any project at the first sign it is escalating. If so, the organization may often kill good projects prematurely, experiencing loss and committing what Keil and his colleagues (Keil et al., 2000) called a "Type 1" error. On the other hand, allowing too many cases of escalation to be continued on the grounds the project may be incorrectly classified (a "Type 2" error) causes loss by wasting valuable resources. These results suggest that managers tend to be too risk-aversive to runaway projects. Consequently, we believe managers should include this new information in determining an appropriate balance between Type 1 and Type 2 errors.

SUMMARY AND CONCLUSION

IMPLICATIONS FOR PRACTICE

In summary, we believe this study's primary contribution to the practice of IS project management is the evidence it provides concerning the prevalence of project over-commitment in both government and private industry in the 21st century. The results suggest most large IS projects will exceed their original budgets and timelines by more than 50%, and much more often in government than in private industry. Second, the study provides evidence corroborating Keil et al. (2000) that "runaway" projects occur frequently, and new empirical evidence that they occur more often in government. Furthermore, the study supports other research that suggests "runaway" projects should be treated fundamentally differently from merely problematic projects, in that more radical organizational changes are needed to reduce management commitment ((Pan et al., 2006a, 2006b; Wright & Capps, 2010). In addition, the results imply managers should consider being more aggressive in promptly shutting down projects that show signs of over-commitment before they needlessly waste valuable resources. The results suggest mangers prematurely kill escalating projects only about 20% of the time. Finally, the results support the idea that nothing is more important in promoting successful IS development project outcomes than best project management practices.

Our survey respondents consistently rated project management factors more important than top management factors in determining project success. Furthermore they rated formal organizational factors more important than informal social or psychological factors. (See Tables 5 and 6).

These particular results underscore the importance of good monitoring of projects. And as Keil et al. (2000) suggest: "To minimize the problems associated with project escalation, managers would do well to implement early warning systems aimed at detecting escalation as early as possible. One way to minimize budget and schedule escalation is to define the de-escalation trigger points at the outset of the project. In this way, when the cost and schedule begin to approach the predefined trigger points, managers can take steps to de-escalate the project and contain the damage. Another related tactic is defining termination conditions at the outset of the project" (Keil et al., 2000). There is a wealth of information on the best practices in IS software development published by Carnegie Mellon's Software Engineering Institute (http://www.sei.cmu.edu/) and the University of Texas' software quality institute (http://lifelong.engr.utexas.edu/sqi/index.cfmj (See also Wright & Capps, 2008).

To sum up, our study's data does not support a conclusion that IT governance measures implemented by many large organizations in the early 21st century have measurably improved IS development project performance. On the contrary, our results support the Standish Group 2009 report, which indicated that such projects are performing worse than they did in the 20th century.

IMPLICATIONS FOR FUTURE RESEARCH

The most interesting research questions this exploratory study raises are related to why IS development projects "run away" or fail significantly more often in government than in private industry. The results indicate that all three project types--failed, problematic, and runaway--are much more likely to occur in government than industry. Is this because the deep structure of government organizations is more resistant to change than that of the private sector? Such a result would be consistent with that predicted by punctuated equilibrium theory (See Table 7). Other unanswered questions related to punctuated equilibrium theory are whether government projects are more often "over-committed" than those in private industry; and, what are the conditions under which incremental organizational changes are preferable to radical changes? Our results suggest that under certain conditions an incremental change, education, is more beneficial than a more radical change, replacing staff (See Tables 9 and 10). However, for "runaway" projects, question 3.8 (See Table 11), this pattern was reversed: the respondents believed it more beneficial to replace project management staff rather than simply educate them.

Because project failure is a politically sensitive subject, and thus difficult to study empirically, many more in-depth case studies are needed, with an aim of generating an explanatory theory of project escalation and de-escalation cycles (Pan et al., 2006a, 2006b; Wright & Capps, 2010). Our results support the claim others have made that further insights can and will come from punctuated equilibrium theory (Eldredge & Gould, 1972; Tushman & Anderson, 1986; Mokyr, 1990; Gersick, 1991; Wright & Capps, 2010) as well as from other organizational theories (Sutton, 1987; Eisenhardt, 1989; Ancona, 1990; Isabella, 1990; Pettigrew, 1990; Elsbach & Sutton, 1992; Shenhar, 1998; Cule & Robey, 2004).

What's more we believe there is a need for the IS audit and academic community to join hands in conducting further pragmatic research. Global surveys of IS auditors concerning the frequency, circumstances and characteristics of problematic IS development projects would be helpful. There are several advantages of relying on IS auditors for this type of research. As reported by Keil et al. (2000) these include: "... IS auditors do not have directly vested interests in project outcomes because their careers are unlikely to be made or broken by a project's success or failure; IS auditors can be expected to report more objectively than managers and other project participants; IS auditors have access to objective data on project performance; and IS auditors have experience with multiple projects and formal standards for judging projects" (Keil et al., 2000). Unfortunately, the Information Systems Audit and Control Association in recent years has reduced its support for global survey research.

The outsourcing literature (See Tiwana & Bush, 2007) also promises to be of help in the study of IS development project performance. The relationship between control practices and the project commitment escalation process is an additional promising area for future research (See Kirsch, 1997; Mahring, 2002; Kirsch, 2004; Tiwana & Bush, 2007). Another area for research would be to go beyond the results of Mahring (2002) and Kirsch (2004) to explore whether the constitution and effectiveness of Information Technology Governance is inversely related to management over-commitment to projects. Specifically, it would be of interest to determine if the organization's ability to adapt control practices to the dynamic nature of complex projects reduces the likelihood of escalation periods. The relationship between over-commitment and the requirements determination process is also a promising one for future research (See Kirsch & Haney, 2006).

LIMITATIONS OF THE STUDY

Because our research was limited to proxy measurements consisting of the beliefs of self-reported information systems (IS) auditors, the conclusions drawn herein must be interpreted with caution. Admittedly, this method raises the possibility of bias or an error in the data set. It is conceivable auditors are more likely to be assigned to troubled projects and thus their estimates of the frequency of project problems may be biased upward. Conversely, IS auditors may tend to be employed by organizations more aware of the need for good project management. If so, it is possible that IS auditors' estimates of project problems may be biased downward. Further, this study's data may have been more reliable if it had been gathered from specific individual projects.

However, such an approach was not deemed appropriate or necessary for an initial exploratory study. Because the study relied on self-reported information concerning past events, the results are also limited by possible recall bias. However the same research method has been used in similar research (i.e., Keil et al., 2000) and we know of no effective methods for avoiding subject recall bias. Despite these limitations of the methodology, we believe this study contains important contributions to research and practice of information systems development.

REFERENCES

Abernathy, W. & Utterback, J.M. (1982). Patterns of industrial innovation. In M. Tushman and W. Moore (eds.) Readings in the Management of Innovation, 97-108. New York: Harper Business.

Abdel-Hamid, T. & Madnick, S.E. (1991). Software project dynamics: an integrated approach. Englewood Cliffs, NJ: Prentice-Hall, Inc.

Ancona, D. (1990). Outward bound: strategies for team survival in an organization. Academy of Management Journal, 33 (2), 334-365.

Anderson, P. & Tushman, M.L. (1990). Technological discontinuities and dominant designs. Administrative Science Quarterly, 35, 604-633.

Auditor General of Canada (2002). Report of the Auditor General of Canada to the House of Commons. Toronto: Office of the Auditor General. Be skeptical about what can be achieved by IS developments. Pessimism, Computer Failure, and Information Systems Development 927 www.oag- vg.gc.ca/domino/reports.nsf/html/02menu_e.html.

Bozman, J. S. (1994). California kills failed $ 44M project. Computerworld, May 9, 1994.

Brockner, J. (1992). The escalation of commitment to a failing course of action: toward theoretical progress. New York: Wiley and Sons.

Collins, T. & Bicknell D. (1997). Crash: learning from the world's worst computer disasters. New York: Simon & Schuster.

Corner, I. & Hinton M. (2002). Customer relationship management systems: implementation risks and relationship dynamics. Qualitative Market Research, 5 (4), 239-51.

Cule, P. & Robey, D. (2004). A dual-motor, constructive process model of organizational transition. Organizational Studies, 25 (2), 229-260.

Dalcher, D. & Audley, G. (2003). Avoiding IS/IT implementation failure. Technology Analysis and Strategic Management, 15 (4), 403-7.

Demarco, T. (1982). Controlling software projects. New York: Yourdon Press.

The Economist (2002). The health service's IT problem. October 19, 2002, pages 51-52.

Eldredge, N. & Gould, S. (1972). Punctuated equilibria: an alternative to phyletic gradualism, in T. J. Schopf (eds.), Models in paleobiology, pages 82-115. San Francisco: Freeman, Cooper and Co.

Elsbach, K.D. & Sutton, R.I. (1992). Acquiring organizational legitimacy through illegitimate actions: a marriage of institutional and impression management theories. Academy of Management Journal, 699-738.

Festinger, L.A. (1957). Theory of Cognitive Dissonance. Evanston, IL: Row, Peterson Publishers.

Garland, H. & Conlon, D. E. (1998). Too close to quit: the role of project completion in maintaining commitment. Journal of Applied Social Psychology, 28 (22), 2025-2048.

Georgiadou, E. (2003). Software process and product improvement: a historical perspective. Cybernetics and Systems Analysis, 39 (1), 125-42.

Gersick, C. (1991). Revolutionary change theories: a multilevel exploration of the punctuated equilibrium paradigm. Academy of Management Review, 16 (1), 10-36.

Glass, R. L. (1998). Software runaways. Upper Saddle River, NJ: Prentice-Hall, Inc.

Goldfinch, S. (2007). Pessimism, Computer failure and information systems development in the public sector. Public Administration Review. September/October 2007.

Heeks, R. (1999). Reinventing government in the information age: international practice in IT-enabled public sector reform. London: Routledge.

Heeks, R. (2002). Failure, success and improvisation of information system projects in developing countries. Development Informatics Working Paper Series, No. 11/2002. Manchester, UK: Institute for Development Policy and Management. Public Administration Review. September/October 2007.

Heeks, R. (2004). eGovernment as a carrier of context. iGovernment Working Paper Series, No. 15. Manchester UK: Institute for Development Policy and Management, University of Manchester.

Heng, Tan, B. & Wei, K. (2003). De-escalation of commitment in software projects: who matters? what matters? Information and Management, 41, 99-110.

Hirschheim, R., Klein, H.K. & Newman, M. (1991). Information systems development as social action: theoretical perspective and practice, OMEGA, 19 (6), 587-608.

Isabella, L.A. (1990). Evolving interpretations as a change unfolds: how managers construe key organizational events. Academy of Management Journal, 33 (1), 7-41.

Iacovou, C. (1999). The IPACS project: when IT hits the fan. Journal of Information Technology 14 (2), 267-275.

James, G. (1997). IT fiascoes and how to avoid them. Datamation, November 1997, pages 84-88.

Johnson, J. (1995). Chaos: the dollar drain IT project failures. Application Development Trends, 2 (1), 41-47.

Johnston H.R. & Carrico S.R. (1988). Developing capabilities to use information strategically. MIS Quarterly, 12 (1) , 37-48.

Keil, M. (1995). Pulling the plug: software project management and the problem of project escalation. MIS Quarterly, 19 (4), 421-447.

Keil, M. & Robey, D. (1999). Turning around troubled software projects: an exploratory study of the de-escalation of commitment to failing courses of action. Journal of Management Information Systems, 15 (4), 63-87.

Keil, M., Mann, J. & Rai, A. (2000). Why software projects escalate: an empirical analysis and test of four theoretical models, MIS Quarterly, 24 (4), 631-664.

Keil, M., Rai, A., Mann, J. & Zhang, G. P. (2003). Why software projects escalate: the importance of project management constructs. IEEE Transactions on Engineering Management, 50, pages 251-261.

Kirsch L.J. (1997). Portfolios of control modes and IS project management. Information Systems Research, 15, 374-395.

Kirsch, L. J. (2004). Deploying common systems globally: the dynamics of control. Information Systems Research, 15 (4), 374-395.

Kirsch, L. J. & Haney, M. H. (2006). Requirements determination for common systems: turning a global vision into a local reality. The Journal of Strategic Information Systems, 15 (2), 79-104.

Knorr, E. (2005). Anatomy of an IT disaster: how the FBI blew it, Infoworld.com. March 21, 2005.

Mahring, M. (2002). IT project governance. Economic Research Institute (EFI). Stockholm School of Economics: Stockholm, Sweden.

Mahring, M. & Keil, M. (2008). Information technology project escalation: a process model. Decision Sciences, 39 (2) , 239-272.

Mann, J. (2003). Preventing runaway IT projects: protecting auditors from entrapment. The Proceedings of Americas Conference on Information Systems, August 4-5, 2003 in Tampa, FLA.

Montealegre, R., & Keil, M. (2000). De-Escalating information technology projects: lessons learned from the Denver international airport. MIS Quarterly, 24 (3), 417-447.

Mokyr, J. (1990). The lever of riches. Oxford UK: Oxford University Press.

Newman, M. & Sabherwal, R. (1996). Commitment determinants to information systems development: a longitudinal investigation. MIS Quarterly, 20, 23-54.

Orlikowski, W.J. (1993). CASE tools as organizational change: investigating incremental and radical changes in systems development. MIS Quarterly, 17 (3), 309-340.

Pan, S.L., Pan, S.C., Newman, M. & Flynn, D. (2006a). Escalation and de-escalation for commitment to information systems projects: insights from a project evaluation model. European Journal of Operational Research, 17 (3), 1139-1160.

Pan, S.L., Pan, S.C., Newman, M. & Flynn, D. (2006b). Escalation and de-escalation for commitment: a commitment transformation analysis of an e-government project. Information Systems Journal, 16, 3-21.

Pettigrew, A.M. (1990). Longitudinal field research on change: theory and practice. Organization Science, 1 (3), 267-292.

Rainer, R. & Turban, E. (2009). Huge problems at Britain's national health system. In Introduction to Information Systems, (pages 321-323). Hoboken, NJ: John Wiley & Sons.

Ross, R.J. & Staw, B. M. (1986). Expo86: an escalation prototype. Administrative Science Quarterly, 3 (1), 274-297.

Ross, R.J. & Staw, B. M. (1993). Organizational escalation and exit: lessons from the Shoreham Nuclear Power Plant. Academy of Management Journal, 36 (4), 701-732.

Royal Academy of Engineering, and British Computing Society (2004). The Challenges of Complex IT Projects. London: Royal Academy of Engineering.

Royer, I. (2002). Escalation in organizations: the role of collective belief. Academy of Management Conference held in Denver, CO.

Royer, I. (2003). Why bad projects are so hard to kill. Harvard Business Review, February, 2003, pages 49-56.

Sabherwal, R. & King, W.R. (1992). Decision processes for developing strategic Apps of IS: a contingency approach. Decision Sciences, 23 (4), 917-943.

Schmidt, J. B. & Calantone, R. J. (2002). Escalation of commitment during new product development. Journal of the Academy of Marketing Science, 30 (2), 103-118.

SIMPL (2000). SIMPL Group, and New Zealand Institute of Economic Research (NZIER), 2000. Information Technology Projects: Performance of the New Zealand Public Sector in Performance. Report to the Department of the Prime Minister and Cabinet. Wellington.

Standish Group (2001). Extreme Chaos. http://standishgroup.com/sample_research/register.php.

Standish Group (2004). Third Quarter Report. http://standishgroup.com/sample_research/register.php.

Standish Group (2009). CHAOS Summary 2009. http://standishgroup.com/sample_research/register.php.

Staw, B.M. (1997). "The escalation of commitment: an update and appraisal," Chapter 9 in Organizational Decision Making. Z. Shapira (eds.). Cambridge, UK: Cambridge University Press, pages 191-215.

Staw, B. M. & Ross, J. (1987). Behavior in escalation situations: antecedents, prototypes and solutions. Research in Organizational Behavior, pages 39-78. B. M. Staw and L. L. Cummings (eds.), JAI Press Inc. of Greenwich, CT.

Sutton, R.I. (1987). The process of organizational death: disbanding and reconnecting. Administrative Science Quarterly, 32 (4), 542-569.

Tiwana, A. & Bush, A. A. (2007). A comparison of transaction cost, agency, and knowledge-based predictors of IT outsourcing decisions: a U.S.-Japan cross cultural field study. Journal of Management Information Systems, 24 (1), 259-300.

Tushman M. & Anderson A. (1986). Technological discontinuities and organizational environments. Administrative Science Quarterly, 31, 439-465.

Tushman, M. & Romanelli, E. (1985). Organizational evolution: a metamorphosis model of convergence and reorientation. In L. L, Cummings and B. M. Staw (eds.), Research in organizational behavior, pages 171- 222.

Tushman, M. L., Newman, W. H. & Romanelli, E. (1986). Convergence and upheaval: managing the unsteady pace of organizational evolution. California Management Review, 29 (1), 29-44.

Tversky, A. & Kahneman, D. (1981). Framing of decisions and the psychology of choice. Science, 211, 453-458.

US GAO. (2006). Weak controls over trilogy project led to payment of questionable contractor costs and missing assets, GAO Report to Congressional Requesters. Government Accountability Office. GAO-06-306 February 2006.

Utterback, J.M. (1994). Radical innovation and corporate regeneration. Research Technology Management, 37 (4), 10-18.

Vitale, M., Ives, B. & Beath, C.M. (1986). Linking information technology and corporate strategy: an organizational view. Proceedings of the Seventh International Conference on Information Systems. San Diego, CA; 265-276.

Whyte, G. (1986). Escalating commitment to a course of action: a reinterpretation. Academy of Management Review, 11 (2), 311-321.

Whyte, G., & Fassina, N. E. (2007). Escalating commitment in group decision-making. Proceedings of the Academy of Management, l-6. Philadelphia: August 2007.

Wilcocks, L. (1994). Managing information systems in U.K. public administration: issues and prospects. Public Administration, 72 (1), 13--32.

Wright, M.K. & Capps, C.J. (2008). Information technology customer service: 'Best Practices' processes for operations. Journal of Applied Business Research, 24 (3), 63-76.

Wright, M.K. & Capps, C.J. (2010). Runaway information technology projects: a punctuated equilibrium analysis. International Journal of Information Technology Project Management, 1 (4) October 2010.

Zmud, R. W. (1980). Management of large software efforts. MIS Quarterly, 4, 45-55.

Keith Wright, University of Houston Downtown

Charles Capps, Sam Houston State University

Table 1: Length of Participant Professional Experience

Question #   Category                         Responses

3.1          More than one year experience       87%
3.2          More than 15 years                  48%
3.3          Between 5 and 15 years              45%

Table 2: Type of Participant Industry Experience

Question #   Category              Responses

3.3          Financial Services     22.4 %
             US State Government     4.7 %
             National Government     8.2 %
             Manufacturing           31.8%
             Trade                  14.1 %
             Health Care            10.6 %
             Transportation          8.2 %

Table 3: Participant Credentials

Question #   Category                                        Responses

3.4          CIA (Certified Internal Auditor)                 28.6 %

             CISSP (Certified Information Systems Security    23.4 %
             Professional)

             CISA (Certified Information Systems Auditor)     77.8 %

             PMP (Project Management Professional)            26.9 %

Table 4: Perceived Overall Project Performance

Measurement Construct       Survey      Response Choice    Responses
                           Question #

Project failure rate          2.1       Greater than 50%     62.5%
                                         Less than 50%       37.5%
Problematic project rate      2.5       Greater than 50%     65.0%
                                         Less than 50%       35.0%
Problematic projects          3.5       Greater than 50%     ** 66%
become runaway                           Less than 50%       ** 33%

Rate at which runaways        3.10       Less than 20%       71.6%
are turned around                       Greater than 20%     28.4%

Premature project             3.12       Less than 20%       55.4%
termination rate                         More than 20%       44.6%

** Indicates that differences in the proportions are not
statistically significant

Table 5: Perceived Causes of Problematic Projects

Survey       Response Choice             Most likely    Least likely
Question #                                 cause          cause

2.7          Formal attributes of the       68.0 %         31.7 %
             project team (size,
             structure, skills, etc.)

             Formal attributes of the        63%            34 %
             surrounding enterprise
             (size, structure, skills,
             etc.)

             Informal social                31.0 %         68.3 %
             relationships among the
             project stakeholders

             Psychology of individual        48 %           50 %
             project stakeholders
             (beliefs, fears,
             motivations, confidence,
             etc.)

Table 6: Perceived Causes of Runaway Projects

               Response Choice                Most           Least
                                           likely cause   likely cause

Survey         Formal attributes of the        75%            25%
Question 3.9   project team (size,
               structure, skills, etc.)        63%            37%
               Formal attributes of the
               surrounding enterprise
               (size, structure, skills,
               etc.)

               Informal social                 27%            73%
               relationships among the
               project stakeholders

               Psychology of individual
               project stakeholders            45%            55%
               (beliefs, fears,
               motivations, confidence,
               etc.)

Table 7: Punctuated Equilibrium: the "Deep Structure" Theorem

8.1   If an organizational structure is controlled by elected
      government officials, its "deep structure" is harder to change
      than that of private industry.

8.2   If the deep structure" of an organization is relatively harder
      to change, it will be relatively more likely to produce
      over-committed projects.

8.3   If management is over-committed to a project, it is more
      likely to runaway or fail.

8.4   Therefore runaway projects and failed projects should be
      expected to occur more often in government than in the private
      sector, all other things equal.

Table 8: IS Project Performance: Government versus Private Sector

Measurement construct             Survey      Government   Private
                                 Question #     Sector     Sector

Most likely sector for              2.6         71.3 %      28.8%
problematic projects

Most likely sector for failed       2.3          78 %        21%
projects

Most confident sector               2.2         11.2 %       89%

Most likely sector for runaway      3.11        80.0 %      20.0%
projects

Table 9: Punctuated Equilibrium: the Revolutionary Changes Theorem

9.1    The "deep structure" of an organization consists of more
       members of top management than of project management.

9.2    The deep structure of an organization is more likely to be
       changed by a radical (or revolutionary) change than by a more
       modest (or incremental) change (Gersick, 1991).

9.3    Replacing top management constitutes a more radical
       organizational change than does educating top management.

9.4    Radical organizational changes are more effective in correcting
       runaway projects than are incremental changes. (From 8.2, 8.3
       and 10.1--10.3)

Table 10: Correcting Problematic Projects

Survey Question #   Response Choices                Responses

3.1                 Improving top management          39.1%
                    Improving project management      60.9%
3.2                 Replacing top management          4.3%
                    Educating top management          95.7%
3.3                 Replacing project management      39.1%
                    Educating project management      60.9%

Table 11: Correcting Runaways Projects

Survey Question #   Response Choices                Responses

3.6                 Improving top management          36.2%
                    Improving project management      63.8%
3.7                 Replacing top management          18.8%
                    Educating top management          81.2%
3.8                 Replacing project management      68.1%
                    Educating project management      31.9%

Table 12: Perceived Causes of Problematic Projects

Survey       Response Choice                    % Responses
Question #

3.5          Rate that problematic        Greater than    Less than
             projects become "runaways"    50% ** 60%     50% ** 40%

3.12         Premature runaway             Less than     Greater than
             termination rate              20% ** 57%     20% ** 43%

2.4          Percent involved with a      Yes? ** 60%     No? ** 40%
             "problematic" projects

3.4          Percent involved with a        Yes? 38%       No? 62%
             "runaway" project

** Indicates that differences in the proportions are not
statistically significant