Identification and ranking of software project risks.

Sharma, Arpita ; Gupta, Aayushi ; Khilnani, Deepti 等

Introduction

It has almost been a decade since the software industry has detonated. It has witnessed a remarkable growth since and has shown a tremendous escalation not only in the core activities but also in the IT enabled services. Despite the uninterrupted expansion, the software industry still has the highest number of project failures. According to Standish report (2005), the total number of projects failed during 2004 was 71%, which is quite exponential. Boehm (1991) and Jiang et al., (1998) found that 15-35% of all the software projects were cancelled outright while remaining projects suffer either from schedule slippage, cost overruns or failure to meet the project goals. Software development projects have a dismal track-record of cost and schedule overruns with quality and usability problems (Kwak et al., 2004). Further, it becomes very difficult to predict the success of project because the scope of the project keeps changing depending upon the market; hence the resources have to be re-allocated leading to schedule slippage and cost overruns. Many software projects involve multiple entities such as companies, divisions, etc., that may have certain interests. There is often a feeling of disconnection between software developers and their management, each believing that the others are out of touch with reality resulting in misunderstanding and lack of trust (Kwak et al., 2004).

Apparently, this implies that software project development is extremely risky. Therefore, managing the involved risks is of primary importance in software project development, especially in large-scale software projects (Yong et al., 2006). If the risks are not controlled at the early stages of the project, it would result in an exponential increase in the cost of the project (Brooks, 1995; Mangione, 2003). But as the returns from the project remains fixed, it leads to an increased probability of project failure as shown in figure 1 given by Mangione (2003).While there are many factors that may result in software failure, lack of risk management is undoubtedly one of the leading factors. It is the multiplicity of risks inherent in software project management that leads to project failure.

[FIGURE 1 OMITTED]

The literature has produced a number of conceptual frameworks to explain different types of software development risk, risk management strategies and measures of software project performance (Keil et al., 2004; Costa et al., 2006). Most of these studies address software risk control in developed countries having sophisticated software development infrastructures (Na et al., 2007). According to Boehm (1991), personnel shortfalls; unrealistic schedules and budgets; and developing wrong functions and properties are the top three risks that a software company faces. Keil et al., (1998) in his study identified the following as the key risk factors: lack of top management support; failure to gain user commitment; and misunderstanding of requirements. Addison et al., (2002) conducted a three phase Delphi survey of the South Africa software industry and identified unclear objectives; misunderstanding the requirements; and failure to gain user involvement as the key risk factors. In a similar study conducted by Smith et al., (2006) in Africa, lack of management support to the project; unclear objectives; and schedule flaw came out to be the major risk factors affecting the software project. Sherer (1995) used a three dimensional framework to explain the software risks and concluded that technical risks were the most important risks affecting the software projects. A study done in India revealed that beside the cost over runs and schedule slippage, the personnel turnover is the biggest risk that the Indian companies are facing (Boehm et al., 1997). According to the study conducted by Arora et al., (2001), the Indian software industry is not only facing the problem of attrition from the home front but is also facing problem from the client side as well. Other major risks included resistance within the US to foreign programmers, poor telecommunication infrastructure, and the delays in obtaining the required visas for Indian programmers (Arora et al., 2001).

As revealed in the literature review, a lots of work has been done on software risks factors and top ten risks that affect the software projects in their respective countries have been identified. With every new research a new risk has been added to the existing list. The present study aims to present a global scenario of the risks being faced by the industry as a whole. The study consolidate and rank the risks identified by various researchers in their respective countries and present the top ten risks affecting the software projects in the global scenario. It also examines the relationship between the top risks identified in the study.

Identification and Ranking of Risks

Based on Sherer's (1995) three dimensional framework, the risk affecting the software projects is analysed. According to the framework, the three dimensions of software risks are technical, organisational and environmental. Technical risks pertain to the uncertainty in tasks and procedures; organisational risks are related to poor communication and organisational structure; and finally environmental risks include rapidly changing environment and problems with external relationships with software developers. For this study, the risks were broadly classified in ten categories namely:

(i) Personnel-inadequate skills and inability to perform the requisite task

(ii) Schedule-inaccurate estimation of schedule or inability to meet the time lines

(iii) Process--lack of effective processes and quality control

(iv) Functionality-failure to understand client requirements

(v) Performance-inability of software to meet the real time performance needs

(vi) Reliability-failure of system due to poor quality of software

(vii) Safety and security-software failure or unauthorised access resulting in monetary loss or loss to life

(viii) User/client involvement-lack of client support and conflict amongst themselves

(ix) Financial-inability of software to generate forecasted cash flows

(x) Maintainability-software in adaptable to the changing environment and unportable to other installations

Numerous studies have been conducted in the past to identify the risks that the software projects face or are likely to face. However in this study, only twelve studies have been considered for further analysis. These studies have been selected on the basis of the extent of coverage of the topic, the in-depth analysis and the objectivity in the paper. In all the twelve studies selected, the risks were identified through a detailed analysis using sophisticated techniques like Delphi method, factor analysis of a field survey. The paper analysis the risks so identified by the researchers by putting each of the risk in the three dimensional framework suggested by Sherer (1995).

The step-by-step procedure of the analysis is as follows:

(i) Refer the study under consideration and take the top ten risks identified in the study one by one.

(ii) Put the risk chosen in the Sherer's (1995) framework according to the risk category it belongs to and the risk dimension (technical, organisational or environmental) it falls into. For instance, "lack of top management" relates to the organisational dimension of risk and falls under process. "Unclear or misunderstood objectives" can be associated with technical dimension and functionality aspect while "lack of available skilled personnel" can be colligated to technical personnel cell.

(iii) Repeat step 2, until all the top ten risks of the study under consideration gets exhausted.

(iv) As there are three dimensions (columns) and ten risks (rows) in total, the resulting framework will be 3 x 10 matrix.

(v) The relative ranking of the three dimensions, according to the study under consideration, is calculated as RDj = [10.summation over (i=1)] Xij/10 where i refer to the rows (i.e the risks) and can have value from 1 to 10; j refer to the columns (i.e. the dimensions) and can have values of 1 to 3. Xij refers to the number of risks belonging to the ith row and jth column in the matrix. For instance, while analysing Smith's et al. (2006) dimension factors, it has been found that more importance is given to the technical dimension, RD1 = 0.4 while organisational and environmental dimensions has 0.3 value respectively.

(vi) The relative ranking of the ten risks, according to the study under consideration, is similarly calculated as RRi = [[10.summation over (i=1)] Xij/10] For instance, while analysing Smith's et al. (2006) risk factors, it has been found that process RR3 = 1 is most important for the success of the project followed by functionality RR4 = 0.67 while the rest of the risk variables have got a value of 0.33 or 0.

(vii) Repeat steps 1-6 till all the twelve studies have been considered. Table 2 gives the relative rankings of the three dimensions of the risks as considered by different researchers. Similarly, Table 3 represents the relative ranking of the ten risk categories.

(viii) The scores of the relative ranks, as calculated in earlier steps for 12 researchers and as given in Table 2, are added. The resultant total is divided by the grand total of all the total values to calculate the relative importance of each of the dimension. From table 2, the individual total of technical, organizational and environmental dimension is coming as 5.95, 3.38 and 2.65 respectively. The relative importance of technical dimension is thus calculated as = (5.95/ (5.95+3.38+2.65)) * 100 = 49%. Similarly, the relative importance of rest of the dimensions can be calculated. The importance of the technical dimension of risk is calculated to be 49%. While organisational and environmental dimensions are just 28% and 22% respectively. It clearly shows that for the success of the project it is more important that the tasks and procedures are well in place. This can further be justified with increase in the number of companies getting CMM level and PCMM certification.

(ix) The relative importance of different risks category has been calculated using the same procedure as given in step 8. The last row of Table 3 shows the relative importance of each category of risks. On the basis of the scores attained by each category of risk in table 3, the top ten risks have been ranked in Table 4. The results thus obtained were shared with some of the senior IT professionals working in Noida. According to the discussions held with the professionals, the top five risks identified in the study are examined below:

Rank I--Functionality Risk

Functionality Risk refers to the system or software not meeting or understanding the user requirements. This is one of the most crucial factors for the success of any software projects. Historically, risk management efforts have been focused on this risk by developing tools and techniques which enable the developer to understand the requirements. These tools are of help only when the initial requirements are fairly constant. But in today's ever changing environment these tools also fail to manage this risk. It has been found, through the interviews conducted and literature reviews, that most of the projects fail because of the failure to comprehend the client requirements or misunderstanding them all together.

Rank II--Process Risk

This risk signifies that if the company has inadequate/ineffective internal and external processes or quality control measures the chances of failure of the project tends to increase manifold. Before CMM and PCMM certifications were introduced the software companies did not have a well established processes thus leading to a failure in developing and delivering the system on time. But today companies are realizing the importance of well placed processes and quality control measures; this can be seen with the increasing number of companies going for these certifications.

Rank III--User or Client Involvement Risk

Sometimes client itself fails to freeze its requirements or define the requirements to the developer. A number of times the conflicts amongst the clients themselves results in a failure of the project due to either escalation of costs or schedule.

Rank IV--Personnel Risk

Personnel risk is one of the biggest risks that the Software Industry has been facing since the very beginning. Shortage of skilled software engineers, low morale of the team, high employee turnover and cultural and lingual differences with the client has caused many projects to see their doom.

Rank V--Schedule Risk

Sometimes it becomes difficult to estimate accurate time lines for the project. This may be due to the flaw in project planning and management or in the organisational processes. Increasing complexity in the software projects also sometimes results in slippage from the schedule. Whatever maybe the cause, this risk is extremely crucial especially when failure to meet the timelines may lead to huge loss of wealth or life.

Relationship between the Risks

[FIGURE 2 OMITTED]

The interviews conducted also revealed that there are important relationships that exist between the top five risks identified in the study. These relationships are shown in Figure 2. As already identified, the two most crucial risks affecting the software project are functionality and process. If these two risks are not managed properly, they will result in increased personnel risk and client risk. If the organisation's processes are not well-defined, the chances are that the top management may not give adequate support to the team and also the team may face lots of internal conflicts. This will lead to low morale in the team thus leading to high attrition rate. Eventually, it will further lead to client risk as the client too will start feeling insecure and disinterested. The increased personnel risk and client risk will result in slippage in time lines and cost, thus increasing the chances of failure of the project. Thus, it is important for companies to develop effective risk management strategies to protect themselves from loss of money, time and reputation due to project failure.

Conclusion

On the basis of the previous 12 research studies conducted across the globe in different countries, top ten risks affecting the software industry have been identified. According to the analysis, the two top most risks are functionality and process. Proper management of these two risks is necessary for the success of the project as these may have crippling effect on the other aspects of the project, which may eventually lead to the failure of the project. The paper has a lot of significance in present day IT scenario concerning the software project success and failures.

References

Addison, T & Vallabh, S (2002), "Controlling software project risks: An empirical study of methods used by experienced project managers", Proceedings of SAICSIT 2002, September, port Elizabeth, pp. 128-140.

Arora, A, Arunachalam, VS, Asundi, J, & Fernandes, R (2001), "The Indian software services industry", Research Policy, Vol.30 (8), pp. 1267-1287.

Boehm, B (1991), "Software risk management principles and practices", IEEE Software Vol. 8 (1), pp. 32-41.

Boehm, BW & DeMarco, T (1997), "Software Risk management", IEEE Software Vol. 14 (3), pp. 17-19.

Brooks, FP (1995), The Mythical Man-month: Essays on Software Engineering, Addison Wesley Longman, United States.

Costa, HR, Barros, MDO & Travassos, GH (2006), "Evaluating software project portfolio risks", Journal of Systems and software, Vol. 80, pp. 16-31.

Field, T (1997), "When BAD things Happen to GOOD projects", CIO, pp. 55-62.

Jiang, J & Klein, G (1998), "Systems analysis attitude toward information systems development", Information Resource Management Journal, Vol. 11 (4), pp. 5-10.

Jiang J & Klein, G (2001), "Software project risks and development focus", Project management Journal, Vol. 32 (1), pp. 20-26.

Keil, M, Cule, P, Lyytinen, K & Schmidt, R (1998), "A framework for identifying software project risks", Communications of the ACM, Vol. 41 (11), pp. 76-83.

Keil, M, Smith, J, Pawlowski, S, & Jin, L (2004), "Why didn't somebody tell me? Climate, information asymmetry, and bad new about troubled projects", DATABASE for advance in Information Systems, Vol. 35 (2), pp. 65-84.

Kwak, YH & Stoddard, J (2004), "Project risk management: lessons learned from software development environment", Technovation, Vol. 24, pp. 915-920.

Mangione, C (2003), "Software Project Failure: The Reasons, The Costs", viewed August 05, 2008, <http://www.cioupdate.com/reports/article.php/1563701>.

Na, KS, Simpson, JT, Li, X, Singh, T & Kim, KY (2007), "Software development risk and project performance measurement: Evidence in Korea", The Journal of systems and software, Vol. 80, pp. 596-605.

Oz, E & Sosik, J (2000), "Why information systems projects are abandoned: a leadership and communication theory and exploratory study", Journal of Computer Information Systems, Fall, pp. 66-78.

Rai, A, Keil, M, Cheney, JE, Zhang, GP (2003), "Why software projects escalate: the importance of project s", management constructs", IEEE Transaction on Engineering Management, Vol. 50 (3), pp. 251-261.

Schmidt, R, Lyytinen, K, Keil, M and Cule, P (2001), "Identifying software project risks: An international Delphi study", Journal of Management Information Systems, Vol. 17 (4), pp. 5-36.

Sherer SA (1995), "The Three dimensions of software risk: Technical, Organisational, and Environmental", Presented in 28th Annual Hawaii International conference on system sciences, pp. 369-378.

Smith, D, Eastcroft, M, Mahmood, N & Rode, H (2006), "Risk factors affecting software projects in South Africa", South African Journal of Business Management, Vol. 3 (2), pp. 55-65.

The Standish 2005--CHAOS report, viewed on June 05, 2008,<http://www.standishgroup.com/quarterly_reports /pdf_copy/q3_2005_sample.pdf >.

Tiwana, A and Keil, M (2006), "Functionality Risk in Information Systems Development: An empirical investigation", IEEE Transaction on Engineering Management, Vol. 53 (3), pp. 412-425 .

Yong, H, Juhua, C, Zhenbang, R, Mei, L & Xie, K (2006), "A Neural Networks Approach for Software Risk Analysis", Sixth IEEE International Conference on Data Mining--Workshops (ICDMW'06).

Arpita Sharma, E-mail: arpita.sharma@jiit.ac.in

Aayushi Gupta, E-mail: aayushi.gupta@jiit.ac.in

Deepti Khilnani, E-mail: deeptikhilnani@gmail.com

Jaypee Institute of Information Technology University, Noida, (U.P.), India.

Table 2: Analysis of the three dimensions of risk

 Technical Organisational Environmental

Smith et al. (2006) 0.4 0.3 0.3
Schmidt et al. (2001) 0.33 0.22 0.44
Boehm (1991) 0.5 0 0.5
Keil et al. (1998) 0.36 0.27 0.36
Jiang et al. (2001) 0.5 0.375 0.125
De Marco et al. (2003) 0.67 0.17 0.17
Oz et al. (2000) 0.4 0.6 0
Addison et al. (2002) 0.44 0.44 0.11
Field (1997) 0.5 0.3 0.2
Tiwana et al. (2006) 0.6 0.2 0.2
Rai et al. (2003) 0.75 0 0.25
Costa et al.(2006) 0.5 0.5 0
 49% 28% 22%

Table 3: Analysis of the ten categories of risk

 Pers Sch Pro Func Perf

D Smith et al., (2006) 0.3 0.3 1 0.6 0.3
Schmidt et al., (2001) 0.3 0 0.6 0.6 0.3
Boehm (1991) 0.3 0.3 0 0.6 0.3
Keil et al., (1998) 1 0 1 0.6 0.3
Jiang et al., (2001) 0.6 0 0.3 0.3 0
De Marco et al., (2003) 0.3 0.3 0.3 0.3 0.3
Oz et al., (2000) 0.3 0.3 0.3 0.3 0
Addison et al., (2002) 0.3 0.6 0.3 0.6 0
Addison et al., (2002) 0.3 0.6 0.3 0.6 0
Field (1997) 0.3 0.3 1 0.6 0.3
Tiwana et al., (2006) 0.3 0 0.3 0.6 0
Rai et al., (2003) 0 0.3 0.3 0.3 0
Costa et al., (2006) 0.33 0 0.3 0 0.3
 14.86% 8.49% 19.15% 19.24% 7.40%

 Rel Saf Us In Fin Main

D Smith et al., (2006) 0 0 0.3 0.3 0
Schmidt et al., (2001) 0 0 1 0 0
Boehm (1991) 0.6 0 0.3 0 0
Keil et al., (1998) 0 0 1 0 0
Jiang et al., (2001) 0 0 0.6 0.6 0
De Marco et al., (2003) 0.3 0 0 0 0
Oz et al., (2000) 0 0 0 0.3 0
Addison et al., (2002) 0.3 0 0.3 0.3 0
Addison et al., (2002) 0.3 0 0.3 0.3 0
Field (1997) 0 0 0.3 0.3 0
Tiwana et al., (2006) 0 0 0.3 0 0
Rai et al., (2003) 0 0 0 0 0.3
Costa et al., (2006) 0.6 0 0.6 0 0.3
 6.40% 0 15.98% 6.37% 2.11%

Pers: personnel, Sch: Schedule, Pro: Process, Func:
Functionality, Perf: Performance, Rel: Reliability,
Saf: Safety, Us Inv: User/Client Involvement,
Fin: Financial, Main: Maintainability

Table 4: Ranking of the top ten risks

Risks Rank

Functionality I
Process II
User/Client Involvement III
Personnel IV
Schedule V
Performance VI
Reliability VII
Financial VIII
Maintainability IX
Safety/ Security X